How can I sanitize a string for use as a filename?

Question

I've got a routine that converts a file into a different format and saves it. The original datafiles were numbered, but my routine gives the output a filename based on an internal name found in the original.

I tried to batch-run it on a whole directory, and it worked fine until I hit one file whose internal name had a slash in it. Oops! And if it does that here, it could easily do it on other files. Is there an RTL (or WinAPI) routine somewhere that will sanitize a string and remove invalid symbols so it's safe to use as a filename?

Answer 1

You can use PathGetCharType function, PathCleanupSpec function or the following trick:

  function IsValidFilePath(const FileName: String): Boolean;
  var
    S: String;
    I: Integer;
  begin
    Result := False;
    S := FileName;
    repeat
      I := LastDelimiter('\/', S);
      MoveFile(nil, PChar(S));
      if (GetLastError = ERROR_ALREADY_EXISTS) or
         (
           (GetFileAttributes(PChar(Copy(S, I + 1, MaxInt))) = INVALID_FILE_ATTRIBUTES)
           and
           (GetLastError=ERROR_INVALID_NAME)
         ) then
        Exit;
      if I>0 then
        S := Copy(S,1,I-1);
    until I = 0;
    Result := True;
  end;

This code divides string into parts and uses MoveFile to verify each part. MoveFile will fail for invalid characters or reserved file names (like 'COM') and return success or ERROR_ALREADY_EXISTS for valid file name.

---

PathCleanupSpec is in the Jedi Windows API under Win32API/JwaShlObj.pas

Answer 2

Regarding the question whether there is any API function to sanitize a file a name (or even check for its validity) - there seems to be none. Quoting from the comment on the PathSearchAndQualify() function:

There does not appear to be any Windows API that will validate a path entered by the user; this is left as an an ad hoc exercise for each application.

So you can only consult the rules for file name validity from File Names, Paths, and Namespaces (Windows):

• Use almost any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:

  • The following reserved characters are not allowed:
    < > : " / \ | ? *
  • Characters whose integer representations are in the range from zero through 31 are not allowed.
  • Any other character that the target file system does not allow.

• Do not use the following reserved device names for the name of a file: CON, PRN, AUX, NUL, COM1..COM9, LPT1..LPT9.
  Also avoid these names followed immediately by an extension; for example, NUL.txt is not recommended.

If you know that your program will only ever write to NTFS file systems you can probably be sure that there are no other characters that the file system does not allow, so you would only have to check that the file name is not too long (use the MAX_PATH constant) after all invalid chars have been removed (or replaced by underscores, for example).

A program should also make sure that the file name sanitizing has not lead to file name conflicts and it silently overwrites other files which ended up with the same name.

Answer 3

{
  CleanFileName
  ---------------------------------------------------------------------------

  Given an input string strip any chars that would result
  in an invalid file name.  This should just be passed the
  filename not the entire path because the slashes will be
  stripped.  The function ensures that the resulting string
  does not hae multiple spaces together and does not start
  or end with a space.  If the entire string is removed the
  result would not be a valid file name so an error is raised.

}

function CleanFileName(const InputString: string): string;
var
  i: integer;
  ResultWithSpaces: string;
begin

  ResultWithSpaces := InputString;

  for i := 1 to Length(ResultWithSpaces) do
  begin
    // These chars are invalid in file names.
    case ResultWithSpaces[i] of 
      '/', '\', ':', '*', '?', '"', '<', '>', '|', ' ', #$D, #$A, #9:
        // Use a * to indicate a duplicate space so we can remove
        // them at the end.
        {$WARNINGS OFF} // W1047 Unsafe code 'String index to var param'
        if (i > 1) and
          ((ResultWithSpaces[i - 1] = ' ') or (ResultWithSpaces[i - 1] = '*')) then
          ResultWithSpaces[i] := '*'
        else
          ResultWithSpaces[i] := ' ';

        {$WARNINGS ON}
    end;
  end;

  // A * indicates duplicate spaces.  Remove them.
  result := ReplaceStr(ResultWithSpaces, '*', '');

  // Also trim any leading or trailing spaces
  result := Trim(Result);

  if result = '' then
  begin
    raise(Exception.Create('Resulting FileName was empty Input string was: '
      + InputString));
  end;
end;

Answer 4

For anyone else reading this and wanting to use PathCleanupSpec, I wrote this test routine which seems to work... there is a definate lack of examples on the 'net. You need to include ShlObj.pas (not sure when PathCleanupSpec was added but I tested this in Delphi 2010) You will also need to check for XP sp2 or higher

procedure TMainForm.btnTestClick(Sender: TObject);
var
  Path: array [0..MAX_PATH - 1] of WideChar;
  Filename: array[0..MAX_PATH - 1] of WideChar;
  ReturnValue: integer;
  DebugString: string;

begin
  StringToWideChar('a*dodgy%\filename.$&^abc',FileName, MAX_PATH);
  StringToWideChar('C:\',Path, MAX_PATH);
  ReturnValue:= PathCleanupSpec(Path,Filename);
  DebugString:= ('Cleaned up filename:'+Filename+#13+#10);
  if (ReturnValue and $80000000)=$80000000 then
    DebugString:= DebugString+'Fatal result. The cleaned path is not a valid file name'+#13+#10;
  if (ReturnValue and $00000001)=$00000001 then
    DebugString:= DebugString+'Replaced one or more invalid characters'+#13+#10;
  if (ReturnValue and $00000002)=$00000002 then
    DebugString:= DebugString+'Removed one or more invalid characters'+#13+#10;
  if (ReturnValue and $00000004)=$00000004 then
    DebugString:= DebugString+'The returned path is truncated'+#13+#10;
  if (ReturnValue and $00000008)=$00000008 then
    DebugString:= DebugString+'The input path specified at pszDir is too long to allow the formation of a valid file name from pszSpec'+#13;
  ShowMessage(DebugString);
end;

Answer 5

// for all platforms (Windows\Unix), uses IOUtils.
function ReplaceInvalidFileNameChars(const aFileName: string; const aReplaceWith: Char = '_'): string;
var
  i: integer;
begin
  Result := aFileName;
  for i := Low(Result) to High(Result) do
  begin
    if not TPath.IsValidFileNameChar(Result[i]) then
      Result[i] := aReplaceWith;
  end;
end.

Answer 6

Well, the easy thing is to use a regex and your favourite language's version of gsub to replace anything that's not a "word character." This character class would be "\w" in most languages with Perl-like regexes, or "[A-Za-z0-9]" as a simple option otherwise.

Particularly, in contrast to some of the examples in other answers, you don't want to look for invalid characters to remove, but look for valid characters to keep. If you're looking for invalid characters, you're always vulnerable to the introduction of new characters, but if you're looking for only valid ones, you might be slightly less inefficient (in that you replaced a character you didn't really need to), but at least you'll never be wrong.

Now, if you want to make the new version as much like the old as possible, you might consider replacement. Instead of deleting, you can substitute a character or characters you know to be ok. But doing that is an interesting enough problem that it's probably a good topic for another question.

Answer 7

Try this on a modern delphi:

 use System.IOUtils;
 ...
 result := TPath.HasValidFileNameChars(FileName, False)

I allows also to have german umlauts or other chars like -, _,.. in a filename.

Answer 8

use this function. work fine for me the purpose is to get back ONE level of directory name

uses shelobj...

function  CleanDirName(DirFileName : String) : String;
var
  CheckStr : String;
  Path: array [0..MAX_PATH - 1] of WideChar;
  Filename: array[0..MAX_PATH - 1] of WideChar;
  ReturnValue: integer;

begin
  //--     The following are considered invalid characters in all names.
  //--     \ / : * ? " < > |

  CheckStr := Trim(DirFileName);
  CheckStr := StringReplace(CheckStr,'/','-',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'\','-',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'.','-',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,':',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'?',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'<',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'>',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'|',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'!',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'~',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'+',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'=',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,')',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'(',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'*',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'&',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'^',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'%',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'$',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'#',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'@',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'{',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'}',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,'"',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,';',' ',[rfReplaceAll, rfIgnoreCase]);
  CheckStr := StringReplace(CheckStr,',',' ',[rfReplaceAll, rfIgnoreCase]);

  // '' become - nil
  CheckStr := StringReplace(CheckStr,'''','',[rfReplaceAll, rfIgnoreCase]);

  StringToWideChar(CheckStr,FileName, MAX_PATH);
  StringToWideChar('C:\',Path, MAX_PATH);
  ReturnValue:= PathCleanupSpec(Path,Filename);

  Filename := StringReplace(Filename,'  ',' ',[rfReplaceAll, rfIgnoreCase]);
  Result := String(Filename);
end;