/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
$this->cli_options[] = array('--excludePath [path-list]', 'Comma separated list of paths to exclude. Example: "uploads/[path1],uploads/[path2],..."');
// Setting up help:
$this->cli_help['name'] = 'lost_files -- Looking for files in the uploads/ folder which does not have a reference in TYPO3 managed records.';
$this->cli_help['description'] = trim('
Assumptions:
- a perfect integrity of the reference index table (always update the reference index table before using this tool!)
- that all contents in the uploads folder are files attached to TCA records and exclusively managed by TCEmain through "group" type fields
- exceptions are: index.html and .htaccess files (ignored)
- exceptions are: RTEmagic* image files (ignored)
- files found in deleted records are included (otherwise you would see a false list of lost files)
The assumptions are not requirements by the TYPO3 API but reflects the de facto implementation of most TYPO3 installations and therefore a practical approach to cleaning up the uploads/ folder.
Therefore, if all "group" type fields in TCA and flexforms are positioned inside the uploads/ folder and if no files inside are managed manually it should be safe to clean out files with no relations found in the system.
Under such circumstances there should theoretically be no lost files in the uploads/ folder since TCEmain should have managed relations automatically including adding and deleting files.
However, there is at least one reason known to why files might be found lost and that is when FlexForms are used. In such a case a change of/in the Data Structure XML (or the ability of the system to find the Data Structure definition!) used for the flexform could leave lost files behind. This is not unlikely to happen when records are deleted. More details can be found in a note to the function TYPO3\\CMS\\Backend\\Utility\\BackendUtility::getFlexFormDS()
Another scenario could of course be de-installation of extensions which managed files in the uploads/ folders.
Automatic Repair of Errors:
- Simply delete lost files (Warning: First, make sure those files are not used somewhere TYPO3 does not know about! See the assumptions above).
');
$this->cli_help['examples'] = '/.../cli_dispatch.phpsh lowlevel_cleaner lost_files -s -r
Will report lost files.';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_options[] = array('--echotree level', 'When "level" is set to 1 or higher you will see the page of the page tree outputted as it is traversed. A value of 2 for "level" will show even more information.');
$this->cli_help['name'] = 'orphan_records -- To find records that has lost their connection with the page tree';
$this->cli_help['description'] = trim('
Assumptions:
- That all actively used records on the website from TCA configured tables are located in the page tree exclusively.
All records managed by TYPO3 via the TCA array configuration has to belong to a page in the page tree, either directly or indirectly as a version of another record.
VERY TIME, CPU and MEMORY intensive operation since the full page tree is looked up!
Automatic Repair of Errors:
- Silently deleting the orphaned records. In theory they should not be used anywhere in the system, but there could be references. See below for more details on this matter.
Manual repair suggestions:
- Possibly re-connect orphaned records to page tree by setting their "pid" field to a valid page id. A lookup in the sys_refindex table can reveal if there are references to a orphaned record. If there are such references (from records that are not themselves orphans) you might consider to re-connect the record to the page tree, otherwise it should be safe to delete it.
');
$this->cli_help['todo'] = trim('
- Implement a check for references to orphaned records and if a reference comes from a record that is not orphaned itself, we might rather like to re-connect the record to the page tree.
- Implement that orphans can be fixed by setting the PID to a certain page instead of deleting.');
$this->cli_help['examples'] = '/.../cli_dispatch.phpsh lowlevel_cleaner orphan_records -s -r
Will report orphan uids from TCA tables.';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_help['name'] = 'rte_images -- Looking up all occurencies of RTEmagic images in the database and check existence of parent and copy files on the file system plus report possibly lost files of this type.';
$this->cli_help['description'] = trim('
Assumptions:
- a perfect integrity of the reference index table (always update the reference index table before using this tool!)
- that all RTEmagic image files in the database are registered with the soft reference parser "images"
- images found in deleted records are included (means that you might find lost RTEmagic images after flushing deleted records)
The assumptions are not requirements by the TYPO3 API but reflects the de facto implementation of most TYPO3 installations.
However, many custom fields using an RTE will probably not have the "images" soft reference parser registered and so the index will be incomplete and not listing all RTEmagic image files.
The consequence of this limitation is that you should be careful if you wish to delete lost RTEmagic images - they could be referenced from a field not parsed by the "images" soft reference parser!
Automatic Repair of Errors:
- Will search for double-usages of RTEmagic images and make copies as required.
- Lost files can be deleted automatically by setting the value "lostFiles" as an optional parameter to --AUTOFIX, but otherwise delete them manually if you do not recognize them as used somewhere the system does not know about.
Manual repair suggestions:
- Missing files: Re-insert missing files or edit record where the reference is found.
');
$this->cli_help['examples'] = '/.../cli_dispatch.phpsh lowlevel_cleaner rte_images -s -r
Reports problems with RTE images';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_help['name'] = 'missing_files -- Find all file references from records pointing to a missing (non-existing) file.';
$this->cli_help['description'] = trim('
Assumptions:
- a perfect integrity of the reference index table (always update the reference index table before using this tool!)
- relevant soft reference parsers applied everywhere file references are used inline
Files may be missing for these reasons (except software bugs):
- someone manually deleted the file inside fileadmin/ or another user maintained folder. If the reference was a soft reference (opposite to a TCEmain managed file relation from "group" type fields), technically it is not an error although it might be a mistake that someone did so.
- someone manually deleted the file inside the uploads/ folder (typically containing managed files) which is an error since no user interaction should take place there.
Automatic Repair of Errors:
- Managed files (TCA/FlexForm attachments): Will silently remove the reference from the record since the file is missing. For this reason you might prefer a manual approach instead.
- Soft References: Requires manual fix if you consider it an error.
Manual repair suggestions:
- Managed files: You might be able to locate the file and re-insert it in the correct location. However, no automatic fix can do that for you.
- Soft References: You should investigate each case and edit the content accordingly. A soft reference to a file could be in an HTML image tag (for example <img src="missing_file.jpg" />) and you would have to either remove the whole tag, change the filename or re-create the missing file.
');
$this->cli_help['examples'] = '/.../cli_dispatch.phpsh lowlevel_cleaner missing_files -s -r
This will show you missing files in the TYPO3 system and only report back if errors were found.';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_help['name'] = 'missing_relations -- Find all record references pointing to a non-existing record.';
$this->cli_help['description'] = trim('
Assumptions:
- a perfect integrity of the reference index table (always update the reference index table before using this tool!)
- all database references to check are integers greater than zero
- does not check if a referenced record is inside an offline branch, another workspace etc. which could make the reference useless in reality or otherwise question integrity
Records may be missing for these reasons (except software bugs):
- someone deleted the record which is technically not an error although it might be a mistake that someone did so.
- after flushing published versions and/or deleted-flagged records a number of new missing references might appear; those were pointing to records just flushed.
Automatic Repair of Errors:
- Only managed references are repaired (TCA-configured).
- Offline Version Records and Non Existing Records: Reference is removed
Manual repair suggestions:
- For soft references you should investigate each case and edit the content accordingly.
- References to deleted records can theoretically be removed since a deleted record cannot be selected and hence your website should not be affected by removal of the reference. On the other hand it does not hurt to ignore it for now. To have this automatically fixed you must first flush the deleted records after which remaining references will appear as pointing to Non Existing Records and can now be removed with the automatic fix.
NOTICE: Uses the Reference Index Table (sys_refindex) for analysis. Update it before use!
');
$this->cli_help['examples'] = '/.../cli_dispatch.phpsh lowlevel_cleaner missing_relations -s -r
Reports missing relations';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
$this->cli_help['name'] = 'syslog -- Show entries from syslog';
$this->cli_help['description'] = trim('
Showing last 25 hour entries from the syslog. More features pending. This is the most basic and can be useful for nightly check test reports.
');
$this->cli_help['examples'] = '';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_options[] = array('--echotree level', 'When "level" is set to 1 or higher you will see the page of the page tree outputted as it is traversed. A value of 2 for "level" will show even more information.');
$this->cli_options[] = array('--pid id', 'Setting start page in page tree. Default is the page tree root, 0 (zero)');
$this->cli_options[] = array('--depth int', 'Setting traversal depth. 0 (zero) will only analyse start page (see --pid), 1 will traverse one level of subpages etc.');
$this->cli_help['name'] = 'cleanflexform -- Find flexform fields with unclean XML';
$this->cli_help['description'] = trim('
Traversing page tree and finding records with FlexForm fields with XML that could be cleaned up. This will just remove obsolete data garbage.
Automatic Repair:
Cleaning XML for FlexForm fields.
');
$this->cli_help['examples'] = '';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_options[] = array('--echotree level', 'When "level" is set to 1 or higher you will see the page of the page tree outputted as it is traversed. A value of 2 for "level" will show even more information.');
$this->cli_options[] = array('--pid id', 'Setting start page in page tree. Default is the page tree root, 0 (zero)');
$this->cli_options[] = array('--depth int', 'Setting traversal depth. 0 (zero) will only analyse start page (see --pid), 1 will traverse one level of subpages etc.');
$this->cli_help['name'] = 'deleted -- To find and flush deleted records in the page tree';
$this->cli_help['description'] = trim('
Traversing page tree and finding deleted records
Automatic Repair:
Although deleted records are not errors to be repaired, this tool allows you to flush the deleted records completely from the system as an automatic action. Limiting this lookup by --pid and --depth can help you to narrow in the operation to a part of the page tree.
');
$this->cli_help['examples'] = '';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_options[] = array('--echotree level', 'When "level" is set to 1 or higher you will see the page of the page tree outputted as it is traversed. A value of 2 for "level" will show even more information.');
$this->cli_options[] = array('--pid id', 'Setting start page in page tree. Default is the page tree root, 0 (zero)');
$this->cli_options[] = array('--depth int', 'Setting traversal depth. 0 (zero) will only analyse start page (see --pid), 1 will traverse one level of subpages etc.');
$this->cli_options[] = array('--flush-live', 'If set, not only published versions from Live workspace are flushed, but ALL versions from Live workspace (which are offline of course)');
$this->cli_help['name'] = 'versions -- To find information about versions and workspaces in the system';
$this->cli_help['description'] = trim('
Traversing page tree and finding versions, categorizing them by various properties.
Published versions from the Live workspace are registered. So are all offline versions from Live workspace in general. Further, versions in non-existing workspaces are found.
Automatic Repair:
- Deleting (completely) published versions from LIVE workspace OR _all_ offline versions from Live workspace (toogle by --flush-live)
- Resetting workspace for versions where workspace is deleted. (You might want to run this tool again after this operation to clean out those new elements in the Live workspace)
- Deleting unused placeholders
');
$this->cli_help['examples'] = '';
}
/**
* Constructor
*
* @todo Define visibility
*/
public function __construct()
{
parent::__construct();
// Setting up help:
$this->cli_help['name'] = 'double_files -- Looking for files from TYPO3 managed records which are referenced more than one time (only one time allowed)';
$this->cli_help['description'] = trim('
Assumptions:
- a perfect integrity of the reference index table (always update the reference index table before using this tool!)
- files found in deleted records are included (otherwise you would see a false list of lost files)
Files attached to records in TYPO3 using a "group" type configuration in TCA or FlexForm DataStructure are managed exclusively by the system and there must always exist a 1-1 reference between the file and the reference in the record.
This tool will expose when such files are referenced from multiple locations which is considered an integrity error.
If a multi-reference is found it was typically created because the record was copied or modified outside of TCEmain which will otherwise maintain the relations correctly.
Multi-references should be resolved to 1-1 references as soon as possible. The danger of keeping multi-references is that if the file is removed from one of the refering records it will actually be deleted in the file system, leaving missing files for the remaining referers!
Automatic Repair of Errors:
- The multi-referenced file is copied under a new name and references updated.
Manual repair suggestions:
- None that can not be handled by the automatic repair.
');
$this->cli_help['examples'] = '/.../cli_dispatch.phpsh lowlevel_cleaner double_files -s -r
This will check the system for double files relations.';
}