/** * Asks for custom mapping * * @param InputInterface $input * @param OutputInterface $output * @param RestoreJob $job * @param RestoreStrategy $strategy * @author Daniel Wendlandt */ private function askForCustomMapping(InputInterface $input, OutputInterface $output, RestoreJob $job, RestoreStrategy $strategy) { $output->write(PHP_EOL); $questions = array(); /** @var Index $index */ foreach ($job->getMappings()->getIndices() as $index) { /** @var Type $type */ foreach ($index->getTypes() as $type) { $mappingAction = $strategy->getMapping($index->getName(), $type->getName()); $targetMapping = null !== $mappingAction ? '<comment>' . $mappingAction->getStrategy() . ': ' . $mappingAction->getTargetIndex() . '/' . $mappingAction->getTargetType() . '</comment>' : '<error>not set</error>'; $questions[] = $index->getName() . '/' . $type->getName() . '[' . $targetMapping . ']'; } } $question = new ChoiceQuestion('<info>Please map all types from backup</info>', $questions); $helper = $this->getHelper('question'); $answerSource = $helper->ask($input, $output, $question); $indexType = explode('/', substr($answerSource, 0, strpos($answerSource, '['))); if (null !== ($existingMappingAction = $strategy->getMapping($indexType[0], $indexType[1]))) { $strategy->removeMappingAction($existingMappingAction); } $mappingAction = new RestoreStrategy\MappingAction(); $mappingAction->setSourceIndex($indexType[0]); $mappingAction->setSourceType($indexType[1]); $mappingAction = $this->askForTargetMapping($input, $output, $mappingAction); $strategy->addMappingAction($mappingAction); if ($job->getMappings()->countTypes() != $strategy->countMappingActions()) { $this->askForCustomMapping($input, $output, $job, $strategy); } }
private function restoreData(RestoreJob $job, JobStats $jobStats, OutputInterface $output) { $memoryAtSection = memory_get_usage(); $timeStartSection = microtime(true); $output->writeln('<info>*** Starting with data restoring ***</info>' . PHP_EOL); $storedStats = array(); /** @var Index $index */ foreach ($job->getMappings()->getIndices() as $index) { $toIndex = null; /** @var Type $type */ foreach ($index->getTypes() as $type) { /** @var Finder $finder */ $finder = $this->filesystem->loadDataFiles($job->getPath(), $index->getName(), $type->getName()); if (null === $finder) { continue; } if (!isset($storedStats[$index->getName()])) { $storedStats[$index->getName()] = array(); } if (!isset($storedStats[$index->getName()][$type->getName()])) { $storedStats[$index->getName()][$type->getName()] = array('filesRead' => 0, 'dataInType' => 0); } $mappingAction = $job->getStrategy()->getMapping($index->getName(), $type->getName()); if (null === $mappingAction) { throw new \Exception('Mapping action missing for "' . $index->getName() . '/' . $type->getName() . '"'); } $docCount = $finder->count(); $output->writeln('<comment>Store Data for:</comment>'); $output->writeln('<comment> [from]' . $index->getName() . '/' . $type->getName() . '</comment>'); $output->writeln('<comment> [to]' . $mappingAction->getTargetIndex() . '/' . $mappingAction->getTargetType() . '</comment>' . PHP_EOL); $progress = new ProgressBar($output, $docCount); $progress->setFormat('debug'); $progress->display(); $progress->start(); /** @var SplFileInfo $file */ foreach ($finder as $file) { $data = json_decode($file->getContents(), true); $id = $data['_id']; $this->elastic->createDocument($mappingAction->getTargetIndex(), $mappingAction->getTargetType(), $id, $data['_source'], $job->getHost(), $job->getPort()); $progress->advance(); $storedStats[$index->getName()][$type->getName()]['filesRead']++; } $progress->finish(); $output->writeln(PHP_EOL); $toIndex = $mappingAction->getTargetIndex(); } $this->elastic->refreshIndex($toIndex, $job->getHost(), $job->getPort()); } //add aggregated to storedStats for comparing later $docStats = $this->elastic->getDocCountByIndexType($job->getHost(), $job->getPort()); foreach ($storedStats as $indexName => $types) { foreach ($types as $typeName => $typeStats) { $storedStats[$indexName][$typeName]['dataInType'] = $docStats->getDocCount($indexName, $typeName); } } $jobStats->setRestoreData(microtime(true) - $timeStartSection, memory_get_usage(), memory_get_usage() - $memoryAtSection, array('storedStats' => $storedStats)); return $storedStats; }