public function preFillFromSourceLanguage($useGoogleTranslateData = true) { $path = APPPATH . "resources/languageforge/semdomtrans/GoogleTranslateHarvester/semdom-google-translate-{$this->languageIsoCode}.txt.gz"; $googleTranslateData = []; if ($useGoogleTranslateData && file_exists($path)) { $lines = gzfile($path); foreach ($lines as $line) { $splitLine = explode("|", $line); if (count($splitLine) == 2) { $googleTranslateData[$splitLine[0]] = $splitLine[1]; } } } // cjh review: we may actually want to only prefill from English, if in the future we allow creating projects from incomplete source projects $sourceProject = new SemDomTransProjectModel($this->sourceLanguageProjectId->asString()); $this->copyXmlToAssets($sourceProject->xmlFilePath); $sourceItems = new SemDomTransItemListModel($sourceProject); $sourceItems->read(); foreach ($sourceItems->entries as $item) { $newItem = new SemDomTransItemModel($this); // if Google translation exists for given name exists, use it if (array_key_exists($item['name']['translation'], $googleTranslateData)) { $newItem->name->translation = $googleTranslateData[$item['name']['translation']]; $newItem->name->status = SemDomTransStatus::Suggested; } // if Google translation exists for given description exists, use it if (array_key_exists($item['description']['translation'], $googleTranslateData)) { $newItem->description->translation = $googleTranslateData[$item['description']['translation']]; $newItem->description->status = SemDomTransStatus::Suggested; } $newItem->key = $item['key']; for ($x = 0; $x < count($item['questions']); $x++) { $q = new SemDomTransQuestion(); // if Google translation exists for given question, use it if (array_key_exists($item['questions'][$x]['question']['translation'], $googleTranslateData)) { $q->question->translation = $googleTranslateData[$item['questions'][$x]['question']['translation']]; $q->question->status = SemDomTransStatus::Suggested; } // if Google translation exists for given question term, use it if (array_key_exists($item['questions'][$x]['terms']['translation'], $googleTranslateData)) { $q->terms->translation = $googleTranslateData[$item['questions'][$x]['terms']['translation']]; $q->terms->status = SemDomTransStatus::Suggested; } $newItem->questions[] = $q; } for ($x = 0; $x < count($item['searchKeys']); $x++) { $sk = new SemDomTransTranslatedForm(); // if Google translation exists for given search key, use it if (array_key_exists($item['searchKeys'][$x]['translation'], $googleTranslateData)) { $sk->translation = $googleTranslateData[$item['searchKeys'][$x]['translation']]; $sk->status = SemDomTransStatus::Suggested; } $newItem->searchKeys[] = $sk; } $newItem->xmlGuid = $item['xmlGuid']; $newItem->write(); } }
/** * Updates the SemdomTransItem - handles logic for switching from needs revision or suggested to draft * @param array $data * @param string $projectId * @return string */ public static function update($data, $projectId) { $projectModel = new SemDomTransProjectModel($projectId); $previousItemModel = new SemDomTransItemModel($projectModel); if ($data["id"] != '') { $previousItemModel->read($data['id']); } $guid = $data["xmlGuid"]; $s = new SemDomTransItemModel($projectModel); $s->xmlGuid = $guid; $s->readByProperty("xmlGuid", $guid); $s->key = $data["key"]; $s->name->translation = $data["name"]["translation"]; $s->name->status = $data["name"]["status"]; if ($previousItemModel->id != '' && $previousItemModel->name->translation != $data["name"]["translation"] && intval($data["name"]["status"]) != SemDomTransStatus::Approved) { $s->name->status = SemDomTransStatus::Draft; } $s->description->translation = $data["description"]["translation"]; $s->description->status = $data["description"]["status"]; if ($previousItemModel->id != '' && $previousItemModel->description->translation != $data["description"]["translation"] && intval($data["description"]["status"]) != SemDomTransStatus::Approved) { $s->description->status = SemDomTransStatus::Draft; } for ($i = 0; $i < count($s->questions); $i++) { $s->questions[$i]->question->translation = $data["questions"][$i]["question"]["translation"]; $s->questions[$i]->question->status = $data["questions"][$i]["question"]["status"]; if ($previousItemModel->id != '' && $previousItemModel->questions[$i]->question->translation != $data["questions"][$i]["question"]["translation"] && intval($data["questions"][$i]["question"]["status"]) != SemDomTransStatus::Approved) { $s->questions[$i]->question->status = SemDomTransStatus::Draft; } $s->questions[$i]->terms->translation = $data["questions"][$i]["terms"]["translation"]; $s->questions[$i]->terms->status = $data["questions"][$i]["terms"]["status"]; if ($previousItemModel->id != '' && $previousItemModel->questions[$i]->terms->translation != $data["questions"][$i]["terms"]["translation"] && intval($data["questions"][$i]["terms"]["status"]) != SemDomTransStatus::Approved) { $s->questions[$i]->terms->status = SemDomTransStatus::Draft; } } for ($i = 0; $i < count($s->searchKeys); $i++) { $s->searchKeys[$i]->translation = $data["searchKeys"][$i]["translation"]; $s->searchKeys[$i]->status = $data["searchKeys"][$i]["status"]; if ($previousItemModel->id != '' && $previousItemModel->searchKeys[$i]->translation != $data["searchKeys"][$i]["translation"] && intval($data["searchKeys"][$i]["status"]) != SemDomTransStatus::Approved) { $s->searchKeys[$i]->status = SemDomTransStatus::Draft; } } $s->write(); return $s->id->asString(); }
public function testEncode_SourceProjectAndTargetProjectHaveItems_DtoAsExpected() { self::$environ = new SemDomMongoTestEnvironment(); self::$environ->cleanPreviousProject('es'); /** @noinspection PhpUnusedLocalVariableInspection */ $sourceProject = self::$environ->getEnglishProjectAndCreateIfNecessary(); $user1Id = self::$environ->createUser('u', 'u', 'u'); $targetProject = self::$environ->createSemDomProject('es', 'Spanish', $user1Id); /* // insert dummy models $sourceItemModel = new SemDomTransItemModel($sourceProject); $sourceItemModel->key = '1'; $sourceItemModel->name = new SemDomTransTranslatedForm('universe'); $sourceItemModel->description = new SemDomTransTranslatedForm('Universe description'); $sq = new SemDomTransQuestion('A universe question', 'A universe question term'); $sourceItemModel->questions = new ArrayOf(function ($data) { return new SemDomTransQuestion(); }); $sourceItemModel->questions[] = $sq; $sourceItemModel->write(); */ $targetItemsModel = new SemDomTransItemListModel($targetProject); $targetItemsModel->read(); $targetItems = $targetItemsModel->entries; $targetItemModel = new SemDomTransItemModel($targetProject); $targetItemModel->readByProperty('xmlGuid', $targetItems[0]['xmlGuid']); $targetItemModel->key = '1'; $targetItemModel->name = new SemDomTransTranslatedForm('wszechswiat'); $targetItemModel->description = new SemDomTransTranslatedForm('Opis wszechswiata'); $tq = new SemDomTransQuestion('Pytanie wszechswiata', 'Termin zwiazany z wszechswiatem'); $targetItemModel->questions = new ArrayOf(function () { return new SemDomTransQuestion(); }); $targetItemModel->questions[] = $tq; $targetItemModel->write(); // call dto //$loadTargetProject = new SemDomTransProjectModel($prId->asString()); //$loadSourceProject = new SemDomTransProjectModel($loadTargetProject->sourceLanguageProjectId); $prId = $targetProject->id; $result = SemDomTransEditDto::encode($prId->asString(), null); // print_r($result); // check dto returns expected results $entries = $result['entries']; $this->assertTrue($entries != null); $this->assertTrue(count($entries) > 0); $firstObject = $entries[0]; $this->assertNotNull($firstObject['key']); $this->assertEquals('1', $firstObject['key']); $this->assertNotNull($firstObject['name']); $this->assertEquals('Universe, creation', $firstObject['name']['source']); $this->assertEquals('wszechswiat', $firstObject['name']['translation']); $this->assertEquals(SemDomTransStatus::Draft, $firstObject['name']['status']); $this->assertNotNull($firstObject['description']); $this->assertEquals('Opis wszechswiata', $firstObject['description']['translation']); $this->assertEquals(SemDomTransStatus::Draft, $firstObject['description']['status']); $this->assertNotNull($firstObject['questions']); $this->assertNotNull($firstObject['questions'][0]); $this->assertNotNull($firstObject['questions'][0]['question']); $this->assertNotNull($firstObject['questions'][0]['terms']); $this->assertEquals('(1) What words refer to everything we can see?', $firstObject['questions'][0]['question']['source']); $this->assertEquals('Pytanie wszechswiata', $firstObject['questions'][0]['question']['translation']); $this->assertEquals('universe, creation, cosmos, heaven and earth, macrocosm, everything that exists', $firstObject['questions'][0]['terms']['source']); $this->assertEquals('Termin zwiazany z wszechswiatem', $firstObject['questions'][0]['terms']['translation']); // this test messes with the English source project self::$environ->clean(); self::$environ->cleanPreviousProject('en'); }
public function testEncode_SourceProjectAndTargetProjectHaveItems_DtoAsExpected() { $e = new SemDomMongoTestEnvironment(); $e->cleanPreviousProject('es'); $sourceProject = $e->getEnglishProjectAndCreateIfNecessary(); $user1Id = $e->createUser('u', 'u', 'u'); $targetProject = $e->createSemDomProject('es', "Spanish", $user1Id); /* // insert dummy models $sourceItemModel = new SemDomTransItemModel($sourceProject); $sourceItemModel->key = "1"; $sourceItemModel->name = new SemDomTransTranslatedForm("universe"); $sourceItemModel->description = new SemDomTransTranslatedForm("Universe description"); $sq = new SemDomTransQuestion("A universe question", "A universe question term"); $sourceItemModel->questions = new ArrayOf(function ($data) { return new SemDomTransQuestion(); }); $sourceItemModel->questions[] = $sq; $sourceItemModel->write(); */ $targetItemsModel = new SemDomTransItemListModel($targetProject); $targetItemsModel->read(); $targetItems = $targetItemsModel->entries; $targetItemModel = new SemDomTransItemModel($targetProject); $targetItemModel->readByProperty('xmlGuid', $targetItems[0]['xmlGuid']); $targetItemModel->key = "1"; $targetItemModel->name = new SemDomTransTranslatedForm("wszechswiat"); $targetItemModel->description = new SemDomTransTranslatedForm("Opis wszechswiata"); $tq = new SemDomTransQuestion("Pytanie wszechswiata", "Termin zwiazany z wszechswiatem"); $targetItemModel->questions = new ArrayOf(function ($data) { return new SemDomTransQuestion(); }); $targetItemModel->questions[] = $tq; $targetItemModel->write(); // call dto //$loadTargetProject = new SemDomTransProjectModel($prId->asString()); //$loadSourceProject = new SemDomTransProjectModel($loadTargetProject->sourceLanguageProjectId); $prId = $targetProject->id; $result = SemDomTransEditDto::encode($prId->asString(), null); // print_r($result); // check dto returns expected results $entries = $result["entries"]; $this->assertTrue($entries != null); $this->assertTrue(count($entries) > 0); $firstObject = $entries[0]; $this->assertNotEqual($firstObject["key"], null); $this->assertEqual($firstObject["key"], "1"); $this->assertNotEqual($firstObject["name"], null); $this->assertEqual($firstObject["name"]["source"], "Universe, creation"); $this->assertEqual($firstObject["name"]["translation"], "wszechswiat"); $this->assertEqual($firstObject["name"]["status"], SemDomTransStatus::Draft); $this->assertNotEqual($firstObject["description"], null); $this->assertEqual($firstObject["description"]["translation"], "Opis wszechswiata"); $this->assertEqual($firstObject["description"]["status"], SemDomTransStatus::Draft); $this->assertNotEqual($firstObject["questions"], null); $this->assertNotEqual($firstObject["questions"][0], null); $this->assertNotEqual($firstObject["questions"][0]["question"], null); $this->assertNotEqual($firstObject["questions"][0]["terms"], null); $this->assertEqual($firstObject["questions"][0]["question"]["source"], "(1) What words refer to everything we can see?"); $this->assertEqual($firstObject["questions"][0]["question"]["translation"], "Pytanie wszechswiata"); $this->assertEqual($firstObject["questions"][0]["terms"]["source"], "universe, creation, cosmos, heaven and earth, macrocosm, everything that exists"); $this->assertEqual($firstObject["questions"][0]["terms"]["translation"], "Termin zwiazany z wszechswiatem"); // this test messes with the English source project $e->clean(); $e->cleanPreviousProject('en'); }
/** * Recursively goes through each node and: * 1) pulls out its values (name, description, questions, search keys) and creates SemDomTransItem from them * 2) Recurses on all direct children (e.g. for 1 we would recurse on 1.1, 1.2, .... 1.5, not 1.1.1) * * Note: all values that are parsed are printed to file storing list of words/phrases/sentences that Google Translate Harvester * can later use. * @param SimpleXMLNode $domainNode * @return SemDomTransQuestion|SemDomTransTranslatedForm */ private function _processDomainNode($domainNode) { $guid = (string) $domainNode['guid']; // retrieve name $name = $this->_getPathVal($domainNode->xpath("Name/AUni[@ws='{$this->_lang}']")); // fwrite($this->_outputFile, $name . "\n"); // retrieve abbrevation $abbreviation = $this->_getPathVal($domainNode->xpath("Abbreviation/AUni[@ws='en']")); //retrieve description $description = $this->_getPathVal($domainNode->xpath("Description/AStr[@ws='{$this->_lang}']")[0]->xpath("Run[@ws='{$this->_lang}']")); // fwrite($this->_outputFile, $description . "\n"); $questions = new ArrayOf(function () { return new SemDomTransQuestion(); }); $searchKeys = new ArrayOf(function () { return new SemDomTransTranslatedForm(); }); // process question if (property_exists($domainNode, 'Questions')) { $questionsXML = $domainNode->Questions->children(); // parse nested questions foreach ($questionsXML as $questionXML) { $question = $this->_getPathVal($questionXML->xpath("Question/AUni[@ws='{$this->_lang}']")); // fwrite($this->_outputFile, $question . "\n"); $terms = $this->_getPathVal($questionXML->xpath("ExampleWords/AUni[@ws='{$this->_lang}']")); // fwrite($this->_outputFile, $terms . "\n"); $q = new SemDomTransQuestion($question, $terms); $sk = new SemDomTransTranslatedForm($terms); // if question is non-empty in XML file, set as approved if ($question != "") { $q->question->status = SemDomTransStatus::Approved; } // if question terms is non-empty in XML file, set as approved if ($terms != '') { $q->terms->status = SemDomTransStatus::Approved; $sk->status = SemDomTransStatus::Approved; } $questions[] = $q; $searchKeys[] = $sk; } } // assume that we are not matching up with guids $itemModel = new SemDomTransItemModel($this->_projectModel); $itemModel->readByProperty('xmlGuid', $guid); $itemModel->xmlGuid = $guid; $itemModel->name = new SemDomTransTranslatedForm($name); // if name is non-empty in XML file, set as approved if ($name != "") { $itemModel->name->status = SemDomTransStatus::Approved; } $itemModel->key = $abbreviation; $itemModel->description = new SemDomTransTranslatedForm($description); // if description is non-empty in XML file, set as approved if ($description != "") { $itemModel->description->status = SemDomTransStatus::Approved; } $itemModel->questions = $questions; $itemModel->searchKeys = $searchKeys; //print_r($itemModel->questions); if ($this->_runForReal) { $itemModel->write(); } //print "Processed $abbreviation $name\n"; // recurse on sub-domains if (property_exists($domainNode, 'SubPossibilities')) { foreach ($domainNode->SubPossibilities->children() as $subDomainNode) { $this->_processDomainNode($subDomainNode); } } }
private function _processDomainNode($domainNode) { if ($this->_xml) { $guid = (string) $domainNode['guid']; } $s = new SemDomTransItemModel($this->_projectModel); $s->readByProperty("xmlGuid", $guid); $abbreviation = $this->_getPathVal($domainNode->xpath("Abbreviation/AUni[@ws='en']")); // if XML file does not have to be recreated, just assign to appropriate fields // if XML has to be recreated, then add appropriate children nodes to XML file if (!$this->_recreateXMLFile) { $name = $domainNode->xpath("Name/AUni[@ws='{$this->_lang}']")[0]; $description = $domainNode->xpath("Description/AStr[@ws='{$this->_lang}']")[0]->xpath("Run[@ws='{$this->_lang}']")[0]; $name[0] = SemDomTransStatus::isApproved($s->name->status) ? $s->name->translation : ''; $description[0] = SemDomTransStatus::isApproved($s->description->status) ? $s->description->translation : ''; } else { $name = $domainNode->Name; $nameChild = clone $domainNode->Name->AUni; $nameChild["ws"] = $this->_lang; $nameChild[0] = SemDomTransStatus::isApproved($s->name->status) ? $s->name->translation : ''; SemDomXMLExporter::_addChild($name, $nameChild); $description = $domainNode->Description; $descriptionChild = clone $domainNode->Description->AStr; $descriptionChild["ws"] = $this->_lang; $descriptionChild->Run["ws"] = $this->_lang; $descriptionChild->Run[0] = SemDomTransStatus::isApproved($s->description->status) ? $s->description->translation : ''; SemDomXMLExporter::_addChild($description, $descriptionChild); } $questions = new ArrayOf(function ($data) { return new SemDomTransQuestion(); }); $searchKeys = new ArrayOf(function ($data) { return new SemDomTransTranslatedForm(); }); if (property_exists($domainNode, 'Questions')) { $questionsXML = $domainNode->Questions; // parse nested questions $index = 0; foreach ($questionsXML->children() as $questionXML) { // if XML file does not have to be recreated, just assign to appropriate fields // if XML has to be recreated, then add appropriate children nodes to XML file if (!$this->_recreateXMLFile) { $question = $this->_getNodeOrNull($questionXML->xpath("Question/AUni[@ws='{$this->_lang}']")); $terms = $this->_getNodeOrNull($questionXML->xpath("ExampleWords/AUni[@ws='{$this->_lang}']")); if ($question != null && SemDomTransStatus::isApproved($s->questions[$index]->question->status)) { $question[0] = $s->questions[$index]->question->translation; } else { $question[0] = ''; } if ($terms != null && SemDomTransStatus::isApproved($s->questions[$index]->question->status)) { $terms[0] = $s->questions[$index]->terms->translation; } } else { $question = $questionXML->Question; $terms = $questionXML->ExampleWords; if (!empty($question)) { $questionChild = clone $question->AUni; $questionChild["ws"] = $this->_lang; if (SemDomTransStatus::isApproved($s->questions[$index]->question->status)) { $questionChild[0] = $s->questions[$index]->question->translation; } else { $questionsChild[0] = ''; } SemDomXMLExporter::_addChild($question, $questionChild); } if (!empty($terms)) { $termsChild = clone $terms->AUni; $termsChild["ws"] = $this->_lang; if (SemDomTransStatus::isApproved($s->questions[$index]->terms->status)) { $termsChild[0] = $s->questions[$index]->question->translation; } else { $termsChild[0] = ''; } $termsChild[0] = $s->questions[$index]->terms->translation; SemDomXMLExporter::_addChild($terms, $termsChild); } } $index++; } } //print "Processed $abbreviation \n"; // recurse on sub-domains if (property_exists($domainNode, 'SubPossibilities')) { foreach ($domainNode->SubPossibilities->children() as $subDomainNode) { $this->_processDomainNode($subDomainNode); } } }