FileInfo.Delete is not deleting the file immediately

Question

I am currently experiencing a serious problem. I am scanning files in a directory, then I process them [read contents with File.ReadAllText(fi.FullName)], and then deleting the file. The thread sleeps for 200 ms, and starts again (scan, process, delete). The issue is that sometimes, I can see that a file that has been already deleted it appears in the next scan, this does not happen always, only occasionally.

List<FileInfo> files = GetFiles();
if (files != null)
{
    foreach (FileInfo fi in files)
    {
       if (ProcessFile(fi))
       {
           fi.Delete();
           log.Info("Report file: " + fi.FullName + " has been deleted");
       }
   }
}

And here is the GetFiles method

internal List<FileInfo> GetFiles()
{
    try
    {
        DirectoryInfo info = new DirectoryInfo(scanDir);
        List<FileInfo> files = info.GetFiles().OrderBy(p => p.CreationTime).Take(10).ToList(); //oldest file first
        return files;
    }
    catch (Exception ex)
    {
       log.Error("Getting files from directory: " + scanDir + ". Error: " + ex.ToString());
        return null;
    }
}

I have read in other posts that the FileInfo.Delete() takes some time, but Microsoft documentation does not say anything about this. So I am not sure as to what is happing. Can anybody spot anything wrong with the code? Is there any official documentation as to whether the fileInfo.Delete() is a blocking call? or does it simple marks a file for deletion?

EDIT and here is the only reference to the FileInfo in the ProcessFile

string message = File.ReadAllText(fi.FullName);

I believe that the File.ReadAllText closes the file, and that no handles should be left around please correct me if wrong!...also, this only happens occasionally, and not to all files (I am processing 10 files, and it happens to just 1)

Answer 1

From the Microsoft Reference Source:

   public override void Delete()
    {
#if FEATURE_CORECLR
        FileSecurityState state = new FileSecurityState(FileSecurityStateAccess.Write, DisplayPath, FullPath);
        state.EnsureState();
#else
        // For security check, path should be resolved to an absolute path.
        new FileIOPermission(FileIOPermissionAccess.Write, new String[] { FullPath }, false, false).Demand();
#endif

        bool r = Win32Native.DeleteFile(FullPath);
        if (!r) {
            int hr = Marshal.GetLastWin32Error();
            if (hr==Win32Native.ERROR_FILE_NOT_FOUND)
                return;
            else
                __Error.WinIOError(hr, DisplayPath);
        }
    }

It is shown that the Delete method uses Win32Native.DeleteFile method in kernel32.dll. And further check on the kernel32.dll can be found in this over 500+ pages reference.

In page 79, you could find reference to DeleteFile:

1.49 DeleteFile

The DeleteFile function deletes an existing file.

DeleteFile: procedure
(
lpFileName: string
);
stdcall;
returns( "eax" );
external( "__imp__DeleteFileA@4" );

Parameters

lpFileName
[in] Pointer to a null-terminated string that specifies the file to be deleted.

Windows NT/2000: In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to nearly 32,000 wide characters, call the Unicode version of the function and prepend "\\?\" to the path. For more information, see File Name Conventions.

Windows 95/98: This string must not exceed MAX_PATH characters.

Return Values

If the function succeeds, the return value is nonzero.
If the function fails, the return value is zero. To get extended error information, call GetLastError.
Remarks
If an application attempts to delete a file that does not exist, the DeleteFile function fails.
To delete or rename a file, you must have either delete permission on the file or delete child permission in the parent directory. 
If you set up a directory with all access except delete and delete child and the ACLs of new files are inherited, then you should be able to create a file without being able to delete it. However, you can then create a file, and you will get all the access you request on the handle returned to you at the time you create the file. 
If you requested delete permission at the time you created the file, you could delete or rename the file with that handle but not with any other.

Windows 95: The DeleteFile function deletes a file even if it is open for normal I/O or as a memory-mapped file. To prevent loss of data, close files before attempting to delete them.

Windows NT/2000: The DeleteFile function fails if an application attempts to delete a file that is open for normal I/O or as a memory-mapped file.

To close an open file, use the CloseHandle function.

MAPI: For more information, see Syntax and Limitations for Win32 Functions Useful in MAPI Development.

Since it is using kernel32.dll, It shares the same mechanism with what we do when we delete file from windows UI.

internal const String KERNEL32 = "kernel32.dll";
...
[DllImport(KERNEL32, SetLastError=true, CharSet=CharSet.Auto, BestFitMapping=false)]
[ResourceExposure(ResourceScope.Machine)]
internal static extern bool DeleteFile(String path);

Thus, this shows that it is not a "blocking" function as you might have suspected. Provided that the file is deletable (there is no access permission error or I/O error) It just takes time to delete.

One workaround for your case would be to try to collect every file which you want to delete first and then delete them together, say, using Async task or something like BackgroundWorker.