Setting Timestamps on files/directories is extremely slow

Question

I'm working on a project, which requires to copy a lot of files and directories, while preserving their original timestamps. So I need to make many calls to the target's SetCreationTime(), SetLastWriteTime() and SetLastAccessTime() methods in order to copy the original values from source to target. As the screenshot below shows, these simple operations take up to 42% of the total computation time.

Since this is limiting my whole application's performance tremendously, I'd like to speed things up. I assume, that each of these calls requires to open and close a new stream to the file/directory. If that's the reason, I'd like to leave this stream open until I finished writing all attributes. How do I accomplish this? I guess this would require the use of some P/Invoke.

Update:

I followed Lukas' advice to use the WinAPI method CreateFile(..) with the FILE_WRITE_ATTRIBUTES. In order to P/Invoke the mentioned method I created following wrapper:

public class Win32ApiWrapper
{
    [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
    private static extern SafeFileHandle CreateFile(string lpFileName,
                                                    [MarshalAs(UnmanagedType.U4)] FileAccess dwDesiredAccess,
                                                    [MarshalAs(UnmanagedType.U4)] FileShare dwShareMode,
                                                    IntPtr lpSecurityAttributes, 
                                                    [MarshalAs(UnmanagedType.U4)] FileMode dwCreationDisposition,
                                                    [MarshalAs(UnmanagedType.U4)] FileAttributes dwFlagsAndAttributes,
                                                    IntPtr hTemplateFile);

    public static SafeFileHandle CreateFileGetHandle(string path, int fileAttributes)
    {
        return CreateFile(path,
                (FileAccess)(EFileAccess.FILE_WRITE_ATTRIBUTES | EFileAccess.FILE_WRITE_DATA),
                0,
                IntPtr.Zero,
                FileMode.Create,
                (FileAttributes)fileAttributes,
                IntPtr.Zero);
        }
}

The enums I used can be found here.This allowed me to do all all things with only opening the file once: Create the file, apply all attributes, set the timestamps and copy the actual content from the original file.

FileInfo targetFile;
int fileAttributes;
IDictionary<string, long> timeStamps; 

using (var hFile = Win32ApiWrapper.CreateFileGetHandle(targetFile.FullName, attributeFlags))
using (var targetStream = new FileStream(hFile, FileAccess.Write))
{
    // copy file
    Win32ApiWrapper.SetFileTime(hFile, timeStamps);
}

Was it worth the effort? YES. It reduced computation time by ~40% from 86s to 51s.

Results before optimization:

Results after optimization:

Answer 1

I'm not a C# programmer and I don't know how those System.IO.FileSystemInfo methods are implemented. But I've made a few tests with the WIN32 API function SetFileTime(..) which will be called by C# at some point.

Here is the code snippet of my benchmark-loop:

#define NO_OF_ITERATIONS   100000

int iteration;
DWORD tStart;
SYSTEMTIME tSys;
FILETIME tFile;
HANDLE hFile;
DWORD tEllapsed;


iteration = NO_OF_ITERATIONS;
GetLocalTime(&tSys);
tStart = GetTickCount();
while (iteration)
{
   tSys.wYear++;
   if (tSys.wYear > 2020)
   {
      tSys.wYear = 2000;
   }

   SystemTimeToFileTime(&tSys, &tFile);
   hFile = CreateFile("test.dat",
                      GENERIC_WRITE,   // FILE_WRITE_ATTRIBUTES
                      0,
                      NULL,
                      OPEN_EXISTING,
                      FILE_ATTRIBUTE_NORMAL,
                      NULL);
   if (hFile == INVALID_HANDLE_VALUE)
   {
      printf("CreateFile(..) failed (error: %d)\n", GetLastError());
      break;
   }

   SetFileTime(hFile, &tFile, &tFile, &tFile);

   CloseHandle(hFile);
   iteration--;
}
tEllapsed = GetTickCount() - tStart;

I've seen that the expensive part of setting the file times is the opening/closing of the file. About 60% of the time is used to open the file and about 40% to close it (which needs to flush the modifications to disc). The above loop took about 9s for 10000 iterations.

A little research showed that calling CreateFile(..) with FILE_WRITE_ATTRIBUTES (instead of GENERIC_WRITE) is sufficient to change the time attributes of a file.

This modification speed things up significantly! Now the same loop finishes within 2s for 10000 iterations. Since the number of iterations is quite small I've made a second run with 100000 iterations to get a more reliable time measurement:

• FILE_WRITE_ATTRIBUTES: 5 runs with 100000 iterations: 12.7-13.2s
• GENERIC_WRITE: 5 runs with 100000 iterations: 63.2-72.5s

Based on the above numbers my guess is that the C# methods use the wrong access mode when opening the file to change to file time. Or some other C# behavior slows things down...

So maybe a solution to your speed issue is to implement a DLL which exports a C function which changes the file times using SetFileTime(..)? Or maybe you can even import the functions CreateFile(..), SetFileTime(..) and CloseHandle(..) directly to avoid calling the C# methods?

Good luck!