protected function execute(InputInterface $input, OutputInterface $output) { $systemCheck = new SystemCheck(); $systemCheck->checkRedisIsInstalled(); $trackerEnvironment = new Environment('tracker'); $trackerEnvironment->init(); Log::unsetInstance(); $trackerEnvironment->getContainer()->get('Piwik\\Access')->setSuperUserAccess(false); $trackerEnvironment->getContainer()->get('Piwik\\Plugin\\Manager')->setTrackerPluginsNotToLoad(array('Provider')); Tracker::loadTrackerEnvironment(); if (OutputInterface::VERBOSITY_VERY_VERBOSE <= $output->getVerbosity()) { $GLOBALS['PIWIK_TRACKER_DEBUG'] = true; } $backend = Queue\Factory::makeBackend(); $queueManager = Queue\Factory::makeQueueManager($backend); if (!$queueManager->canAcquireMoreLocks()) { $trackerEnvironment->destroy(); $this->writeSuccessMessage($output, array("Nothing to proccess. Already max number of workers in process.")); return; } $shouldProcess = false; foreach ($queueManager->getAllQueues() as $queue) { if ($queue->shouldProcess()) { $shouldProcess = true; break; } } if (!$shouldProcess) { $trackerEnvironment->destroy(); $this->writeSuccessMessage($output, array("No queue currently needs processing")); return; } $output->writeln("<info>Starting to process request sets, this can take a while</info>"); register_shutdown_function(function () use($queueManager) { $queueManager->unlock(); }); $startTime = microtime(true); $processor = new Processor($queueManager); $processor->setNumberOfMaxBatchesToProcess(1000); $tracker = $processor->process(); $neededTime = microtime(true) - $startTime; $numRequestsTracked = $tracker->getCountOfLoggedRequests(); $requestsPerSecond = $this->getNumberOfRequestsPerSecond($numRequestsTracked, $neededTime); Piwik::postEvent('Tracker.end'); $trackerEnvironment->destroy(); $this->writeSuccessMessage($output, array(sprintf('This worker finished queue processing with %sreq/s (%s requests in %02.2f seconds)', $requestsPerSecond, $numRequestsTracked, $neededTime))); }
public function test_loadTrackerEnvironment_shouldNotThrow_whenConfigNotFound() { $this->assertTrue(!array_key_exists('PIWIK_TRACKER_DEBUG', $GLOBALS)); $this->assertFalse(SettingsServer::isTrackerApiRequest()); $this->assertTrue(is_readable(Config::getInstance()->getLocalPath())); $this->removeConfigFile(); $this->assertFalse(is_readable(Config::getInstance()->getLocalPath())); Tracker::loadTrackerEnvironment(); $this->assertTrue(SettingsServer::isTrackerApiRequest()); //always reset on the test itself $this->restoreConfigFile(); }
require_once PIWIK_INCLUDE_PATH . '/core/Db/Factory.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Base.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Generic.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Mysql/Generic.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Pgsql/Generic.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Mysql/LogAction.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Pgsql/LogAction.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Mysql/LogVisit.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Pgsql/LogVisit.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Mysql/LogLinkVisitAction.php'; require_once PIWIK_INCLUDE_PATH . '/core/Db/DAO/Pgsql/LogLinkVisitAction.php'; // TODO should move to Tracker application class later. currently needed for environment validation. SettingsServer::setIsTrackerApiRequest(); $environment = new \Piwik\Application\Environment('tracker'); $environment->init(); Tracker::loadTrackerEnvironment(); $tracker = new Tracker(); $requestSet = new RequestSet(); ob_start(); try { $handler = Handler\Factory::make(); $response = $tracker->main($handler, $requestSet); if (!is_null($response)) { echo $response; } } catch (Exception $e) { echo "Error:" . $e->getMessage(); exit(1); } if (ob_get_level() > 1) { ob_end_flush();
/** * The actual task is defined in this method. Here you can access any option or argument that was defined on the * command line via $input and write anything to the console via $output argument. * In case anything went wrong during the execution you should throw an exception to make sure the user will get a * useful error message and to make sure the command does not exit with the status code 0. * * Ideally, the actual command is quite short as it acts like a controller. It should only receive the input values, * execute the task by calling a method of another class and output any useful information. * * Execute the command like: ./console queuedtracking:test --name="The Piwik Team" */ protected function execute(InputInterface $input, OutputInterface $output) { $systemCheck = new SystemCheck(); $systemCheck->checkRedisIsInstalled(); $trackerEnvironment = new Environment('tracker'); $trackerEnvironment->init(); Tracker::loadTrackerEnvironment(); $settings = Queue\Factory::getSettings(); $output->writeln('<comment>Settings that will be used:</comment>'); $output->writeln('Host: ' . $settings->redisHost->getValue()); $output->writeln('Port: ' . $settings->redisPort->getValue()); $output->writeln('Timeout: ' . $settings->redisTimeout->getValue()); $output->writeln('Password: '******'Database: ' . $settings->redisDatabase->getValue()); $output->writeln('NumQueueWorkers: ' . $settings->numQueueWorkers->getValue()); $output->writeln('NumRequestsToProcess: ' . $settings->numRequestsToProcess->getValue()); $output->writeln('ProcessDuringTrackingRequest: ' . (int) $settings->processDuringTrackingRequest->getValue()); $output->writeln('QueueEnabled: ' . (int) $settings->queueEnabled->getValue()); $output->writeln(''); $output->writeln('<comment>Version / stats:</comment>'); $output->writeln('PHP version: ' . phpversion()); $output->writeln('Uname: ' . php_uname()); $extension = new \ReflectionExtension('redis'); $output->writeln('PHPRedis version: ' . $extension->getVersion()); $backend = Queue\Factory::makeBackend(); $output->writeln('Redis version: ' . $backend->getServerVersion()); $output->writeln('Memory: ' . var_export($backend->getMemoryStats(), 1)); $redis = $backend->getConnection(); $evictionPolicy = $this->getRedisConfig($redis, 'maxmemory-policy'); $output->writeln('MaxMemory Eviction Policy config: ' . $evictionPolicy); if ($evictionPolicy !== 'allkeys-lru' && $evictionPolicy !== 'noeviction') { $output->writeln('<error>The eviction policy can likely lead to errors when memory is low. We recommend to use eviction policy <comment>allkeys-lru</comment> or alternatively <comment>noeviction</comment>. Read more here: http://redis.io/topics/lru-cache</error>'); } $evictionPolicy = $this->getRedisConfig($redis, 'maxmemory'); $output->writeln('MaxMemory config: ' . $evictionPolicy); $output->writeln(''); $output->writeln('<comment>Performing some tests:</comment>'); if (method_exists($redis, 'isConnected')) { $output->writeln('Redis is connected: ' . (int) $redis->isConnected()); } if ($backend->testConnection()) { $output->writeln('Connection works in general'); } else { $output->writeln('Connection does not actually work: ' . $redis->getLastError()); } $this->testRedis($redis, 'set', array('testKey', 'value'), 'testKey', $output); $this->testRedis($redis, 'setnx', array('testnxkey', 'value'), 'testnxkey', $output); $this->testRedis($redis, 'setex', array('testexkey', 5, 'value'), 'testexkey', $output); $this->testRedis($redis, 'set', array('testKeyWithNx', 'value', array('nx')), 'testKeyWithNx', $output); $this->testRedis($redis, 'set', array('testKeyWithEx', 'value', array('ex' => 5)), 'testKeyWithEx', $output); $backend->delete('foo'); if (!$backend->setIfNotExists('foo', 'bar', 5)) { $output->writeln("setIfNotExists(foo, bar, 1) does not work, most likely we won't be able to acquire a lock:" . $redis->getLastError()); } else { $initialTtl = $redis->ttl('foo'); if ($initialTtl > 3 && $initialTtl <= 5) { $output->writeln('Initial expire seems to be set correctly'); } else { $output->writeln('<error>Initial expire seems to be not set correctly: ' . $initialTtl . ' </error>'); } if ($backend->get('foo') == 'bar') { $output->writeln('setIfNotExists works fine'); } else { $output->writeln('There might be a problem with setIfNotExists'); } if ($backend->expireIfKeyHasValue('foo', 'bar', 10)) { $output->writeln('expireIfKeyHasValue seems to work fine'); } else { $output->writeln('<error>There might be a problem with expireIfKeyHasValue: ' . $redis->getLastError() . '</error>'); } $extendedTtl = $redis->ttl('foo'); if ($extendedTtl > 8 && $extendedTtl <= 10) { $output->writeln('Extending expire seems to be set correctly'); } else { $output->writeln('<error>Extending expire seems to be not set correctly: ' . $extendedTtl . ' </error>'); } if ($backend->expireIfKeyHasValue('foo', 'invalidValue', 10)) { $output->writeln('<error>expireIfKeyHasValue expired a key which it should not have since values does not match</error>'); } else { $output->writeln('expireIfKeyHasValue correctly expires only when the value is correct'); } $extendedTtl = $redis->ttl('foo'); if ($extendedTtl > 7 && $extendedTtl <= 10) { $output->writeln('Expire is still set which is correct'); } else { $output->writeln('<error>Expire missing after a wrong extendExpire: ' . $extendedTtl . ' </error>'); } if ($backend->deleteIfKeyHasValue('foo', 'bar')) { $output->writeln('deleteIfKeyHasValue seems to work fine'); } else { $output->writeln('<error>There might be a problem with deleteIfKeyHasValue: ' . $redis->getLastError() . '</error>'); } } $redis->delete('fooList'); $backend->appendValuesToList('fooList', array('value1', 'value2', 'value3')); $values = $backend->getFirstXValuesFromList('fooList', 2); if ($values == array('value1', 'value2')) { $backend->removeFirstXValuesFromList('fooList', 1); $backend->removeFirstXValuesFromList('fooList', 1); $values = $backend->getFirstXValuesFromList('fooList', 2); if ($values == array('value3')) { $output->writeln('List feature seems to work fine'); } else { $output->writeln('List feature seems to work only partially: ' . var_export($values, 1)); } } else { $output->writeln('<error>List feature seems to not work fine: ' . $redis->getLastError() . '</error>'); } $output->writeln(''); $output->writeln('<comment>Done</comment>'); }