/** * @param string $entryId * @param FileSync $object * @param int $fileSyncId * @param string $sourceFileUrl * @return BatchJob */ public static function addFileSyncImportJob($entryId, FileSync $fileSync, $sourceFileUrl, BatchJob $parentJob = null, $fileSize = null) { $partnerId = $fileSync->getPartnerId(); $fileSyncId = $fileSync->getId(); $dc = $fileSync->getDc(); KalturaLog::log(__METHOD__ . " entryId[{$entryId}], partnerId[{$partnerId}], fileSyncId[{$fileSyncId}], sourceFileUrl[{$sourceFileUrl}]"); $fileSyncImportData = new kFileSyncImportJobData(); $fileSyncImportData->setSourceUrl($sourceFileUrl); $fileSyncImportData->setFilesyncId($fileSyncId); // tmpFilePath and destFilePath will be set later during get exlusive call on the target data center $batchJob = null; if ($parentJob) { $batchJob = $parentJob->createChild(true, $dc); } else { $batchJob = new BatchJob(); $batchJob->setDc($dc); $batchJob->setEntryId($entryId); $batchJob->setPartnerId($partnerId); } $batchJob->setFileSize($fileSize); KalturaLog::log("Creating Filesync Import job, with file sync id: {$fileSyncId} size: {$fileSize}"); return kJobsManager::addJob($batchJob, $fileSyncImportData, BatchJobType::FILESYNC_IMPORT); }
/** * addFlavorConvertJob adds a single flavor conversion * * @param FileSyncKey $srcSyncKey * @param flavorParamsOutput $flavor * @param int $flavorAssetId * @param int $mediaInfoId * @param BatchJob $parentJob * @param int $lastEngineType * @param BatchJob $dbConvertFlavorJob * @return BatchJob */ public static function addFlavorConvertJob(FileSyncKey $srcSyncKey, flavorParamsOutput $flavor, $flavorAssetId, $mediaInfoId = null, BatchJob $parentJob = null, $lastEngineType = null, BatchJob $dbConvertFlavorJob = null) { $localPath = null; $remoteUrl = null; $flavorAsset = assetPeer::retrieveById($flavorAssetId); if (!$flavorAsset) { KalturaLog::err("No flavor asset found for id [{$flavorAssetId}]"); return null; } if ($flavor->getSourceRemoteStorageProfileId() == StorageProfile::STORAGE_KALTURA_DC) { list($fileSync, $local) = kFileSyncUtils::getReadyFileSyncForKey($srcSyncKey, true, false); $partner = PartnerPeer::retrieveByPK($flavorAsset->getPartnerId()); if (!$fileSync) { kBatchManager::updateEntry($flavorAsset->getEntryId(), entryStatus::ERROR_CONVERTING); $flavorAsset->setStatus(flavorAsset::FLAVOR_ASSET_STATUS_ERROR); $flavorAsset->setDescription("Source file sync not found: {$srcSyncKey}"); $flavorAsset->save(); KalturaLog::err("Source file sync not found: {$srcSyncKey}"); return null; } if (!$local) { if ($fileSync->getFileType() == FileSync::FILE_SYNC_FILE_TYPE_URL && $partner && $partner->getImportRemoteSourceForConvert()) { KalturaLog::debug("Creates import job for remote file sync"); $flavorAsset->setStatus(flavorAsset::FLAVOR_ASSET_STATUS_WAIT_FOR_CONVERT); $flavorAsset->setDescription("Source file sync is importing: {$srcSyncKey}"); $flavorAsset->save(); $originalFlavorAsset = assetPeer::retrieveOriginalByEntryId($flavorAsset->getEntryId()); $url = $fileSync->getExternalUrl($flavorAsset->getEntryId()); return kJobsManager::addImportJob($parentJob, $flavorAsset->getEntryId(), $partner->getId(), $url, $originalFlavorAsset, null, null, true); } throw new kCoreException("Source file not found for flavor conversion [{$flavorAssetId}]", kCoreException::SOURCE_FILE_NOT_FOUND); } if ($fileSync->getFileType() != FileSync::FILE_SYNC_FILE_TYPE_URL) { $localPath = $fileSync->getFullPath(); } $remoteUrl = $fileSync->getExternalUrl($flavorAsset->getEntryId()); } else { $fileSync = kFileSyncUtils::getReadyExternalFileSyncForKey($srcSyncKey, $flavor->getSourceRemoteStorageProfileId()); if (!$fileSync) { kBatchManager::updateEntry($flavorAsset->getEntryId(), entryStatus::ERROR_CONVERTING); $description = "Remote source file sync not found {$srcSyncKey}, storage profile id [" . $flavor->getSourceRemoteStorageProfileId() . "]"; KalturaLog::err($description); $flavorAsset->setStatus(flavorAsset::FLAVOR_ASSET_STATUS_ERROR); $flavorAsset->setDescription($description); $flavorAsset->save(); return null; } $localPath = $fileSync->getFilePath(); $remoteUrl = $fileSync->getExternalUrl($flavorAsset->getEntryId()); } $wamsAssetId = $fileSync->getWamsAssetId(); // creates convert data $convertData = new kConvertJobData(); $convertData->setSrcFileSyncLocalPath($localPath); $convertData->setSrcFileSyncRemoteUrl($remoteUrl); $convertData->setMediaInfoId($mediaInfoId); $convertData->setFlavorParamsOutputId($flavor->getId()); $convertData->setFlavorAssetId($flavorAssetId); $convertData->setSrcFileSyncWamsAssetId($wamsAssetId); KalturaLog::log("Conversion engines string: '" . $flavor->getConversionEngines() . "'"); $currentConversionEngine = null; // TODO remove after all old version flavors migrated // parse supported engine types $conversionEngines = array(); if (!$flavor->getEngineVersion()) { $conversionEngines = explode(',', $flavor->getConversionEngines()); KalturaLog::log(count($conversionEngines) . " conversion engines found for the flavor"); $currentConversionEngine = reset($conversionEngines); // gets the first engine type } // remove until here if (is_null($lastEngineType)) { KalturaLog::log("Last Engine Type is null, engine version [" . $flavor->getEngineVersion() . "]"); if ($flavor->getEngineVersion()) { $operatorSet = new kOperatorSets(); $operatorSet->setSerialized($flavor->getOperators()); $nextOperator = $operatorSet->getOperator(); if (!$nextOperator) { KalturaLog::err("First operator is invalid"); return null; } KalturaLog::log("Set first operator in first set"); $currentConversionEngine = $nextOperator->id; } } else { if ($parentJob && $flavor->getEngineVersion() && ($parentJob->getJobType() == BatchJobType::CONVERT || $parentJob->getJobType() == BatchJobType::POSTCONVERT)) { // using next oprator KalturaLog::log("Adding next conversion operator"); $parentData = $parentJob->getData(); if (!$parentData || !$parentData instanceof kConvartableJobData) { KalturaLog::err("Parent job data is invalid"); return null; } $operatorSet = new kOperatorSets(); $operatorSet->setSerialized($flavor->getOperators()); $nextOperatorSet = $parentData->getCurrentOperationSet(); $nextOperatorIndex = $parentData->getCurrentOperationIndex() + 1; $nextOperator = $operatorSet->getOperator($nextOperatorSet, $nextOperatorIndex); if (!$nextOperator) { KalturaLog::err("Next operator is invalid"); return null; } KalturaLog::log("Moving to next operator [{$nextOperatorIndex}] in set [{$nextOperatorSet}]"); $convertData->setCurrentOperationSet($nextOperatorSet); $convertData->setCurrentOperationIndex($nextOperatorIndex); $currentConversionEngine = $nextOperator->id; } else { // TODO remove after all old version flavors migrated KalturaLog::log("Last used conversion engine is [{$lastEngineType}]"); // searching for $lastEngineType in the list while ($lastEngineType != $currentConversionEngine && next($conversionEngines)) { $currentConversionEngine = current($conversionEngines); } // takes the next engine $currentConversionEngine = next($conversionEngines); if (!$currentConversionEngine) { KalturaLog::err("There is no other conversion engine to use"); return null; } } } if (!is_null($fileSync->getWamsAssetId())) { $currentConversionEngine = conversionEngineType::WAMS; } KalturaLog::log("Using conversion engine [{$currentConversionEngine}]"); // creats a child convert job if (is_null($dbConvertFlavorJob)) { if ($parentJob) { $dbConvertFlavorJob = $parentJob->createChild(); KalturaLog::log("Created from parent convert job with entry id [" . $dbConvertFlavorJob->getEntryId() . "]"); } else { $dbConvertFlavorJob = new BatchJob(); $dbConvertFlavorJob->setEntryId($flavor->getEntryId()); $dbConvertFlavorJob->setPartnerId($flavor->getPartnerId()); $dbConvertFlavorJob->save(); KalturaLog::log("Created from flavor convert job with entry id [" . $dbConvertFlavorJob->getEntryId() . "]"); } } if (!is_null($fileSync->getWamsAssetId())) { $srcFileSize = kWAMS::getInstance($flavor->getPartnerId())->getFileSizeForAssetId($fileSync->getWamsAssetId()); } else { $srcFileSize = kFile::fileSize($convertData->getSrcFileSyncLocalPath()); } $dbConvertFlavorJob->setFileSize($srcFileSize); // TODO remove after all old version flavors migrated if (in_array(conversionEngineType::ENCODING_COM, $conversionEngines)) { $dbConvertFlavorJob->setOnStressDivertTo(conversionEngineType::ENCODING_COM); } // remove until here /* // Remarked by Dor until Tantan's return. // Code is supposed to get a configuration file from the engine and attach it to the batch job. // Was added for document conversion and is not used for now because of a bug of PDFCreator. KalturaLog::log("Calling CDLProceessFlavor with flavor params output[" . $flavor->getId() . "]"); $config = KDLWrap::CDLProceessFlavor($flavor); if($config) { $syncKey = $dbConvertFlavorJob->getSyncKey(BatchJob::FILE_SYNC_BATCHJOB_SUB_TYPE_CONFIG); kFileSyncUtils::file_put_contents($syncKey, $config); $fileSync = kFileSyncUtils::getLocalFileSyncForKey($syncKey); $remoteUrl = $fileSync->getExternalUrl($flavor->getEntryId()); $localPath = kFileSyncUtils::getLocalFilePathForKey($syncKey); $convertData->setConfigLocalPath($localPath); $convertData->setConfigRemoteUrl($remoteUrl); } */ $dbCurrentConversionEngine = kPluginableEnumsManager::apiToCore('conversionEngineType', $currentConversionEngine); return kJobsManager::addJob($dbConvertFlavorJob, $convertData, BatchJobType::CONVERT, $dbCurrentConversionEngine); }
/** * addFlavorConvertJob adds a single flavor conversion * * @param FileSyncKey $srcSyncKey * @param flavorParamsOutput $flavor * @param int $flavorAssetId * @param int $mediaInfoId * @param BatchJob $parentJob * @param int $lastEngineType * @param BatchJob $dbConvertFlavorJob * @return BatchJob */ public static function addFlavorConvertJob(FileSyncKey $srcSyncKey, flavorParamsOutput $flavor, $flavorAssetId, $mediaInfoId = null, BatchJob $parentJob = null, $lastEngineType = null, BatchJob $dbConvertFlavorJob = null) { list($fileSync, $local) = kFileSyncUtils::getReadyFileSyncForKey($srcSyncKey, true, false); $localPath = null; $remoteUrl = null; if ($fileSync) { if ($fileSync->getFileType() != FileSync::FILE_SYNC_FILE_TYPE_URL) { $localPath = $fileSync->getFullPath(); } $remoteUrl = $fileSync->getExternalUrl(); } // creates convert data $convertData = new kConvertJobData(); $convertData->setSrcFileSyncLocalPath($localPath); $convertData->setSrcFileSyncRemoteUrl($remoteUrl); $convertData->setMediaInfoId($mediaInfoId); $convertData->setFlavorParamsOutputId($flavor->getId()); $convertData->setFlavorAssetId($flavorAssetId); KalturaLog::log("Conversion engines string: '" . $flavor->getConversionEngines() . "'"); $currentConversionEngine = null; // TODO remove after all old version flavors migrated // parse supported engine types $conversionEngines = array(); if (!$flavor->getEngineVersion()) { $conversionEngines = explode(',', $flavor->getConversionEngines()); KalturaLog::log(count($conversionEngines) . " conversion engines found for the flavor"); $currentConversionEngine = reset($conversionEngines); // gets the first engine type } // remove until here if (is_null($lastEngineType)) { KalturaLog::log("Last Engine Type is null, engine version [" . $flavor->getEngineVersion() . "]"); if ($flavor->getEngineVersion()) { $operatorSet = new kOperatorSets(); $operatorSet->setSerialized($flavor->getOperators()); $nextOperator = $operatorSet->getOperator(); if (!$nextOperator) { KalturaLog::log("First operator is invalid"); return null; } KalturaLog::log("Set first operator in first set"); $currentConversionEngine = $nextOperator->id; } } else { if ($parentJob && $flavor->getEngineVersion() && ($parentJob->getJobType() == BatchJobType::CONVERT || $parentJob->getJobType() == BatchJobType::POSTCONVERT)) { // using next oprator KalturaLog::log("Adding next conversion operator"); $parentData = $parentJob->getData(); if (!$parentData || !$parentData instanceof kConvartableJobData) { KalturaLog::log("Parent job data is invalid"); return null; } $operatorSet = new kOperatorSets(); $operatorSet->setSerialized($flavor->getOperators()); $nextOperatorSet = $parentData->getCurrentOperationSet(); $nextOperatorIndex = $parentData->getCurrentOperationIndex() + 1; $nextOperator = $operatorSet->getOperator($nextOperatorSet, $nextOperatorIndex); if (!$nextOperator) { KalturaLog::log("Next operator is invalid"); return null; } KalturaLog::log("Moving to next operator [{$nextOperatorIndex}] in set [{$nextOperatorSet}]"); $convertData->setCurrentOperationSet($nextOperatorSet); $convertData->setCurrentOperationIndex($nextOperatorIndex); $currentConversionEngine = $nextOperator->id; } else { // TODO remove after all old version flavors migrated KalturaLog::log("Last used conversion engine is [{$lastEngineType}]"); // searching for $lastEngineType in the list while ($lastEngineType != $currentConversionEngine && next($conversionEngines)) { $currentConversionEngine = current($conversionEngines); } // takes the next engine $currentConversionEngine = next($conversionEngines); if (!$currentConversionEngine) { KalturaLog::log("There is no other conversion engine to use"); return null; } } } KalturaLog::log("Using conversion engine [{$currentConversionEngine}]"); // creats a child convert job if (is_null($dbConvertFlavorJob)) { if ($parentJob) { $dbConvertFlavorJob = $parentJob->createChild(); KalturaLog::log("Created from parent convert job with entry id [" . $dbConvertFlavorJob->getEntryId() . "]"); } else { $dbConvertFlavorJob = new BatchJob(); $dbConvertFlavorJob->setEntryId($flavor->getEntryId()); $dbConvertFlavorJob->setPartnerId($flavor->getPartnerId()); $dbConvertFlavorJob->save(); KalturaLog::log("Created from flavor convert job with entry id [" . $dbConvertFlavorJob->getEntryId() . "]"); } } $dbConvertFlavorJob->setFileSize(filesize($convertData->getSrcFileSyncLocalPath())); // TODO remove after all old version flavors migrated if (in_array(conversionEngineType::ENCODING_COM, $conversionEngines)) { $dbConvertFlavorJob->setOnStressDivertTo(conversionEngineType::ENCODING_COM); } // remove until here /* // Remarked by Dor until Tantan's return. // Code is supposed to get a configuration file from the engine and attach it to the batch job. // Was added for document conversion and is not used for now because of a bug of PDFCreator. KalturaLog::log("Calling CDLProceessFlavor with flavor params output[" . $flavor->getId() . "]"); $config = KDLWrap::CDLProceessFlavor($flavor); if($config) { $syncKey = $dbConvertFlavorJob->getSyncKey(BatchJob::FILE_SYNC_BATCHJOB_SUB_TYPE_CONFIG); kFileSyncUtils::file_put_contents($syncKey, $config); $fileSync = kFileSyncUtils::getLocalFileSyncForKey($syncKey); $remoteUrl = $fileSync->getExternalUrl(); $localPath = kFileSyncUtils::getLocalFilePathForKey($syncKey); $convertData->setConfigLocalPath($localPath); $convertData->setConfigRemoteUrl($remoteUrl); } */ $dbCurrentConversionEngine = kPluginableEnumsManager::apiToCore('conversionEngineType', $currentConversionEngine); return kJobsManager::addJob($dbConvertFlavorJob, $convertData, BatchJobType::CONVERT, $dbCurrentConversionEngine); }