0

During a lengthy and possibly system stressful operation I update a file from time to time via a File Handle and WriteFile

I noticed that when I right mouse click the file and ask for its properties in Windows I don't see it has been updated at all. The Modified time/data doesn't reflect the actual update I did.

I realize that Windows may buffer the content in memory until the file handle is closed. The whole idea is to update the file, so that when the system crashes we at least know where we got (sort of).

How do I make sure the content gets flushed without closing the file handle ?

Peter
  • 1,334
  • 12
  • 28
  • 3
    The data is written, but the directory entry is not updated until the file is closed. You can open another handle to the file and close it, if all you want to do is update the directory. If you want to flush the data, you can use `FlushFileBuffers`. But hard drives are notorious for ignoring flush commands. – Raymond Chen Feb 02 '15 at 03:01
  • _The data is written, but the directory entry is not updated until the file is closed_ Is this something I can count on ? I'm far less interested in the directory update. As long as the actual change in the file really happened. Although, if the file-size grows it should be updated in the directory as well I suppose, otherwise the end of the file is 'forgotten' after a crash. – Peter Feb 02 '15 at 03:46
  • 1
    The file size is correct in the file object. The directory entry is merely advisory; it is not authoritative. Note that the data is written into cache, which flushes at two second intervals. If you want to be sure it is physically on disk, you need to `FlushFileBuffers` and hope you don't have a drive that lies about flushing. – Raymond Chen Feb 02 '15 at 04:14
  • I'm not sure we mean the same with directory entry. I was thinking in terms of file-system (NTFS for instance). If the NTFS file entry is not updated with the new length, then everything that was appended to the file would be lost after a system crash, because the new size would not be known. I'm not sure if `FlushFileBuffers` forces a file-system update ? But I would assume so, if the right-mouse-click file properties shows an updated Modified time ? I thought that's what you meant with directory entry. Good to know the buffer is auto-flushed every 2 seconds. – Peter Feb 02 '15 at 04:43
  • There is FILE_FLAG_WRITE_THROUGH you can use if you want to get your data on disk ASAP. This degrades your performance but does not lazily buffer your data. Typically you use that if you want to make sure that data your application thinks it wrote is actually on disk and not lost when the application crashes before the buffers are flushed. – BitTickler Feb 02 '15 at 04:56
  • 1
    There is a file. That file may appear in multiple directories (due to hard links). Each appearance of the file is a directory entry. Directory entries are not authoritative. They contain "last known good" values which are updated when the handle is closed. If you update a file, only the directory entry that you used to open the file is updated when the handle is closed. The other directory entries are not updated. The file size in the file object is always up to date. The directory entry is not updated until the handle is closed. – Raymond Chen Feb 02 '15 at 06:19

1 Answers1

1

I recommend you either try flushing the buffer or check whether WriteFile is returning TRUE. If it's not, there's probably something wrong, like permission denied or something. You can use GetLastError for that.

André Fratelli
  • 5,920
  • 7
  • 46
  • 87
  • WriteFile() must indeed return true, otherwise there is a problem. I tried `FlushFileBuffers` and I was surprised how long that command needs to complete !? Far longer than when I simply close the Handle ?? That's going to cause a serious performance penalty. – Peter Feb 02 '15 at 03:28
  • 1
    That would be normal, I guess. When you flush a file the operating system writes data to the actual hard drive, which is a very, very slow operation. Buffered I/O exists for this reason. When writing large amounts of data I guess you don't have a choice but to suffer that penalty, however. Did it work? – André Fratelli Feb 02 '15 at 03:33
  • The Modified time/date was indeed updated after the flush. It took far longer than such a write normally requires. Before, when I closed the handle everything would update fast. Now, to test, with a flush before closing the handle, it was very slow. So the Flush seems to be far slower than simply closing the handle ? I have no idea why ? – Peter Feb 02 '15 at 03:43
  • According to the docs: Due to disk caching interactions within the system, the FlushFileBuffers function can be inefficient when used after every write to a disk drive device when many writes are being performed separately. If an application is performing multiple writes to disk and also needs to ensure critical data is written to persistent media, the application should use unbuffered I/O instead of frequently calling FlushFileBuffers. – André Fratelli Feb 02 '15 at 03:45
  • I don't think there's a way around this, to be honest, other than trying a different framework. I have never seen profiling comparisons for this, but I guess you can easily find those on google. Maybe try using standard C functions for writing files, or STL. Again, only by profiling would it be possible to say which is the best approach. – André Fratelli Feb 02 '15 at 03:48
  • I'm going to experiment a bit more with this. Thanks ! – Peter Feb 02 '15 at 04:02
  • Sure thing. Just out of curiosity, let me know what you find =) best. – André Fratelli Feb 02 '15 at 04:03