Typo3 : add file to FAL when it is not presente in sys_file

Question

I have this situation

1. I have a table in my db containing some file names in a field1 (eg field1: "my file.ext")

NOTE: the filename does not necessarily pass a Typo3 "sanitizeFilename" check -> it may contain spaces " " or other characters that would be removed by the sanitizeFilename () method

2. I have the file mentioned above, stored on the server that host typo3

3. In the sys_file table, the file is not present

4. the "update storage index" scheduler cannot process all the files, and if i launch it, it "destroy" the file name (my file.ext -> my_file.ext), so the name stored in the field of my table doensn't have much sense anymore.

I would need to absorb the above mentioned files in the FAL, in order to use them in an ext typo3.

I had thought of such a solution

<?php 
// read from "field1" of my table 
// $filename = the name extracted from my table (e.g. : "my file.ext")
// %path = the path of the file : e.g. "/fileadmin/user_upload")

if (file_exists($_SERVER['DOCUMENT_ROOT'] . "/" . $defaultStorage->getConfiguration()['basePath'] . $path . $filename)) {
    // check the folder
    if ($defaultStorage->hasFolder($path)) {
        $folder = $defaultStorage->getFolder($path);
    } else {
        throw new \ Exception ($path . "path not found in AbstractImportCommand in method extractFile");
    }
    // CHECK IF FILE IS IN FAL
    $file = $folder->getStorage()->getFileInFolder($filename, $folder);
    if ($file) {
        // the file already exists in the FAL
    } else {
      // create new sys_file
      $file = $defaultStorage->addFile(
          $_SERVER['DOCUMENT_ROOT'] . "/" . $defaultStorage->getConfiguration()['basePath'] . $path . $filename, 
          $folder, 
          DuplicationBehavior::REPLACE
      );
    }
}

Any suggestion?

Answer 1

1. Put your code into a command.
2. (optional) create a sys_file_metadata record for your file if you have information that needs to be stored there
3. create a sys_file_reference to your content record (before that you should adjust TCA accordingly)

For creating the sys_file_reference there is no api. A function doing so could look like this:

/**
 * @param $fileUid
 * @param $recordUid
 * @param $table
 */
private function createSysFileReference($record, $fileUid, $tableName, $fieldName){
    $data['sys_file_reference']['NEW_' . uniqid()] = [
        'table_local' => 'sys_file',
        'uid_local' => $fileUid,
        'tablenames' => $tableName,
        'uid_foreign' => $record['uid'],
        'fieldname' => $fieldName,
        'pid' => $record['pid']
    ];

    $dataHandler = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance(\TYPO3\CMS\Core\DataHandling\DataHandler::class);
    $dataHandler->start($data, []);
    $dataHandler->process_datamap();
}

Answer 2

What you consider as "destroying the filename" is indeed preserving file functionality, so sanitizing the filenames is required.

As example consider a file with white space like "My Pdf.pdf". It will be shown eventually like "My%20Pdf.pdf" in the URL and also saved like this. If you link to it, at least the link text won't show the "%20" and the expectation that links and the name of the file are always synchronized (no matter how it's stored) isn't reliable as it probably depends on several parameters like operating system or browser too. The same problem might occur for many different signs too.

Consider that the problems occur not only when a user is downloading a file but also when a file is uploaded, where the wrong url-encoded name is saved in the database, this name might be saved differently in the filesystem or not be found even if the saved value is the same as in the filesystem, due to the url-encoded signs. Your file references are broken on the server then and according links might not work and images not displayed.

So circumventing FAL beside tables is a bad decision and while it's likely possible to write an own sanitizer, I would refrain from dropping it completely.