/** * Funkce pro export pole s informacemi z DataDictionary a TransformationDictionary * @param Datasource $datasource * @param Metasource|null $metasource * @return array */ public function exportDictionariesArr(Datasource $datasource, Metasource $metasource = null) { $this->databasesFacade->openDatabase($datasource->getDbConnection()); $output = array('dataDictionary' => array(), 'transformationDictionary' => array(), 'recordCount' => $this->databasesFacade->getRowsCount($datasource->dbTable)); #region datafields foreach ($datasource->datasourceColumns as $datasourceColumn) { $output['dataDictionary'][$datasourceColumn->name] = $datasourceColumn->type == DatasourceColumn::TYPE_STRING ? 'string' : 'integer'; //TODO kontrola, jaké má smysl vracet datové typy.... } #endregion datafields #region atributy if (!empty($metasource) && !empty($metasource->attributes)) { $this->databasesFacade->openDatabase($metasource->getDbConnection()); foreach ($metasource->attributes as $attribute) { $valuesArr = array(); try { $valuesStatistics = $this->databasesFacade->getColumnValuesStatistic($metasource->attributesTable, $attribute->name, true); if (!empty($valuesStatistics->valuesArr)) { foreach ($valuesStatistics->valuesArr as $value => $count) { $valuesArr[] = $value; } } } catch (\Exception $e) { } $output['transformationDictionary'][$attribute->name] = array('choices' => $valuesArr); } } #endregion atributy return $output; }
/** * Formulář pro import CSV souboru * @return Form */ public function createComponentImportCsvForm() { $form = new Form(); $form->setTranslator($this->translator); $tableName = $form->addText('table', 'Table name:')->setAttribute('class', 'normalWidth')->setRequired('Input table name!'); $presenter = $this; $tableName->addRule(Form::MAX_LENGTH, 'Max length of the table name is %s characters!', 30)->addRule(Form::MIN_LENGTH, 'Min length of the table name is %s characters!', 3)->addRule(Form::PATTERN, 'Table name can contain only letters, numbers and underscore and start with a letter!', '[a-zA-Z0-9_]+')->addRule(function (TextInput $control) use($presenter) { $formValues = $control->form->getValues(true); $csvColumnsCount = $presenter->fileImportsFacade->getColsCountInCSV($formValues['file'], $formValues['separator'], $formValues['enclosure'], $formValues['escape']); $databaseType = $presenter->databasesFacade->prefferedDatabaseType($csvColumnsCount); $newDatasource = $presenter->datasourcesFacade->prepareNewDatasourceForUser($this->usersFacade->findUser($presenter->user->id), $databaseType); $presenter->databasesFacade->openDatabase($newDatasource->getDbConnection()); return !$presenter->databasesFacade->checkTableExists($control->value); }, 'Table with this name already exists!'); $form->addSelect('separator', 'Separator:', array(',' => 'Comma (,)', ';' => 'Semicolon (;)', '|' => 'Vertical line (|)', '\\t' => 'Tab (\\t)'))->setRequired()->setAttribute('class', 'normalWidth'); $form->addSelect('encoding', 'Encoding:', array('utf8' => 'UTF-8', 'cp1250' => 'WIN 1250', 'iso-8859-1' => 'ISO 8859-1'))->setRequired()->setAttribute('class', 'normalWidth'); $file = $form->addHidden('file'); $file->setAttribute('id', 'frm-importCsvForm-file'); $form->addHidden('type'); $form->addText('enclosure', 'Enclosure:', 1, 1)->setDefaultValue('"'); $form->addText('escape', 'Escape:', 1, 1)->setDefaultValue('\\'); $nullValuesArr = CsvImport::getDefaultNullValuesArr(); $defaultNullValue = 'none'; foreach ($nullValuesArr as $value => $text) { $defaultNullValue = $value; break; } $form->addSelect('nullValue', 'Null values:', array_merge(['none' => '--none--'], $nullValuesArr))->setAttribute('title', 'This value will be imported as missing (null).')->setDefaultValue($defaultNullValue)->setAttribute('class', 'normalWidth'); $form->addSubmit('submit', 'Import data into database...')->onClick[] = function (SubmitButton $submitButton) { /** @var Form $form */ $form = $submitButton->form; $values = $form->getValues(); $nullValue = $values->nullValue == 'none' ? null : $values->nullValue; $user = $this->usersFacade->findUser($this->user->id); $colsCount = $this->fileImportsFacade->getColsCountInCSV($values->file, $values->separator, $values->enclosure, $values->escape); $dbType = $this->databasesFacade->prefferedDatabaseType($colsCount); //připravení připojení k DB $datasource = $this->datasourcesFacade->prepareNewDatasourceForUser($user, $dbType); $this->fileImportsFacade->importCsvFile($values->file, $datasource->getDbConnection(), $values->table, $values->encoding, $values->separator, $values->enclosure, $values->escape, $nullValue); $datasource->name = $values->table; //uložíme datasource $this->datasourcesFacade->saveDatasource($datasource); //smažeme dočasné soubory... $this->fileImportsFacade->deleteFile($values->file); $this->redirect('Data:newMinerFromDatasource', array('datasource' => $datasource->datasourceId)); }; $form->addSubmit('storno', 'storno')->setValidationScope([])->onClick[] = function (SubmitButton $button) use($file) { /** @var DataPresenter $presenter */ $presenter = $button->form->getParent(); $this->fileImportsFacade->deleteFile($file->value); $presenter->redirect('Data:newMiner'); }; return $form; }
/** * Formulář pro vytvoření nového formátu * @return Form */ protected function createComponentNewFormatForm() { $form = new Form(); $form->addHidden('datasource'); $form->addHidden('column'); $metaAttribute = $form->addHidden('metaAttribute'); $form->addText('metaAttributeName', 'Meta-attribute:')->setAttribute('readonly', 'readonly')->setAttribute('class', 'normalWidth'); $formatName = $form->addText('formatName', 'Format name:')->setRequired()->setAttribute('class', 'normalWidth'); $formatName->addRule(Form::MIN_LENGTH, 'Min length of format name is %s characters!', 3); $formatName->addRule(function (TextInput $control) use($metaAttribute) { try { $format = $this->metaAttributesFacade->findFormatByName($metaAttribute->value, $control->value); ///XXX if ($format instanceof Format) { return false; } } catch (\Exception $e) { /*chybu ignorujeme (nenalezený metaatribut je OK)*/ } return true; }, 'Format with this name already exists!'); $form->addCheckbox('formatShared', 'Create shared (standard) format'); $form->addSelect('formatType', 'Values range:', array('interval' => 'Continuous values (interval)', 'values' => 'Distinct values (enumeration)'))->setAttribute('class', 'normalWidth')->setDefaultValue('values'); $submit = $form->addSubmit('create', 'Create format'); $submit->setValidationScope(array($formatName)); $submit->onClick[] = function (SubmitButton $button) { $values = $button->form->values; try { $datasourceColumn = $this->datasourcesFacade->findDatasourceColumn($values->datasource, $values->column); $metaAttribute = $this->metaAttributesFacade->findMetaAttribute($values->metaAttribute); $datasource = $this->datasourcesFacade->findDatasource($values->datasource); $this->databasesFacade->openDatabase($datasource->getDbConnection()); $datasourceColumnValuesStatistic = $this->databasesFacade->getColumnValuesStatistic($datasource->dbTable, $datasourceColumn->name); $format = $this->metaAttributesFacade->createFormatFromDatasourceColumn($metaAttribute, $values->formatName, $datasourceColumn, $datasourceColumnValuesStatistic, @$values->formatType, @$values->formatShared); $datasourceColumn->format = $format; $this->datasourcesFacade->saveDatasourceColumn($datasourceColumn); } catch (\Exception $e) { $this->flashMessage($this->translator->translate('Format creation failed.')); } $this->redirect('Close'); }; $storno = $form->addSubmit('storno', 'Storno'); $storno->setValidationScope(array()); $storno->onClick[] = function (SubmitButton $button) { $values = $button->form->values; $this->redirect('SelectFormat', array('datasource' => $values->datasource, 'column' => $values->column, 'metaAttribute' => $values->metaAttribute)); }; $form->onError[] = function (Form $form) { $values = $form->values; $this->handleNewFormat($values->datasource, $values->column, $values->metaAttribute); }; return $form; }
/** * Funkce, která projde všechny hodnoty v daném zdrojovém sloupci a předzpracované je překopíruje do cílové tabulky * @param DatasourceColumn $datasourceColumn * @param Metasource $metasource * @param Attribute $attribute * @param int $attributeStrLen * @param callable $function * @throws \Exception */ private function generateAttributeUsingPreprocessingFunction(DatasourceColumn $datasourceColumn, Metasource $metasource, Attribute $attribute, $attributeStrLen, $function) { $datasource = $datasourceColumn->datasource; $this->databasesFacade->openDatabase($datasource->getDbConnection(), $metasource->getDbConnection()); //vytvoření DB sloupce $dbColumn = new DbColumn(); $dbColumn->name = $attribute->name; $dbColumn->dataType = DbColumn::TYPE_STRING; $dbColumn->strLength = $attributeStrLen; try { $this->databasesFacade->createColumn($metasource->attributesTable, $dbColumn, DatabasesFacade::SECOND_DB); } catch (\Exception $e) { throw new \Exception('Attribute creation failed!', $e); } $valuesArr = $this->databasesFacade->getColumnValuesWithId($datasource->dbTable, $datasourceColumn->name, DatabasesFacade::FIRST_DB); if (!empty($valuesArr)) { $updateDataArr = array(); foreach ($valuesArr as $id => $value) { $updateDataArr[$id] = $function($value); //$this->databasesFacade->updateColumnValueById($metasource->attributesTable,$attribute->name,$id,$function($value),DatabasesFacade::SECOND_DB); } $this->databasesFacade->multiUpdateColumnValueById($metasource->attributesTable, $attribute->name, $updateDataArr, DatabasesFacade::SECOND_DB); } }
public function createMetasourcesTables(Metasource $metasource) { $datasource = $metasource->miner->datasource; $this->databasesFacade->openDatabase($datasource->getDbConnection(), $metasource->getDbConnection()); $attributesTable = $metasource->attributesTable; $i = 1; while ($this->databasesFacade->checkTableExists($attributesTable . DatabasesFacade::SECOND_DB)) { $attributesTable = $metasource->attributesTable . $i; $i++; } $this->databasesFacade->createTable($attributesTable, array(), DatabasesFacade::SECOND_DB); //nakopírování hodnot ID $values = $this->databasesFacade->getColumnValuesWithId($metasource->attributesTable, 'id', DatabasesFacade::SECOND_DB); if (!empty($values)) { foreach ($values as $value) { $this->databasesFacade->insertRow($metasource->attributesTable, array('id' => $value)); } } }