Flush disk write cache

Question

When the policy for a disk in Windows XP and Vista is set to enable write caching on the hard disk, is there a way to flush a file that has just been written, and ensure that it has been committed to disk?

I want to do this programmatically in C++.

Closing the file does perform a flush at the application level, but not at the operating system level. If the power is removed from the PC after closing the file, but before the operating system has flushed the disk write cache, the file is lost, even though it was closed.

Answer 1

.NET FileStream.Flush() will NOT flush the Windows cache for that file content; Flush() only flushes the .NET internal file buffer. In .NET 4.0, Microsoft fixed the problem by adding an optional parameter to Flush() which if set true causes FlushFileSystemBuffers to be called. In .NET 3.5 and below your only choice is to call FlushFileBuffers via pinvoke. See MSDN'sFileStream.Flush community comment for how to do this.

Answer 2

You should not fix this at the time you close the file. Windows will cache, unless you open the file passing FILE_FLAG_WRITE_THROUGH to CreateFile().

You may also want to pass FILE_FLAG_NO_BUFFERING; this tells Windows not to keep a copy of the bytes in cache.

This is more efficient than FlushFileBuffers(), according to the CreateFile documentation on MSDN.

See also file buffering and file caching on MSDN.

Answer 3

You haven't specified the development environment, so:

.Net

IO streams have a .Flush method that does what you want.

Win32 API

There is the FlushFileBuffers call, which takes a file handle as argument.

EDIT (based on a comment from the OA): FlushFileBuffers does not need administrative privileges; it does only if the handle passed to it is the handle for a volume, not for a single file.

Answer 4

You should also note, that your data might not get flushed to the actual disk, even when invoking a flush method of your frameworks API.

Calling the flush method will only tell the kernel to flush its pages to disk. However, if you have the disk write-cache turned on, it is allowed to delay the actual writing process indefinitely.

In order to ensure that your data gets written to the physical layer you have to turn of the write cache in your operating system. This most often comes with a performance penalty up to one or two orders of magnitude when dealing with a lot of small io-operations. Battery based support (UPS) or disks that accept commands to flush the disk write-cache are another option to deal with this problem.

Answer 5

From the microsoft documents you would use _flushall and link in COMMODE.OBJ to ensure that all buffers were committed to disk.

Answer 6

See here: https://jeffpar.github.io/kbarchive/kb/066/Q66052/

When you initially open your file using fopen, include the "c" mode option as the LAST OPTION:

fopen( path, "wc") // w - write mode, c - allow immediate commit to disk

Then when you want to force a flush to disk, call

_flushall()

We made this call before calling

fclose()

We experienced the exact issue you described and this approach fixed it.

Note that this approach does NOT required Administrative rights, which FlushFileBuffers does require, as others have mentioned.

From that above site:

"Microsoft C/C++ version 7.0 introduces the "c" mode option for the fopen() function. When an application opens a file and specifies the "c" mode, the run-time library writes the contents of the file buffer to disk when the application calls the fflush() or _flushall() function. "

Answer 7

You can open/create the file with the FileOptions.WriteThrough flag, which will cause the file to write directly into the disk, bypassing any caches.

E.g.

var file = File.Open(
    "1.txt",
    new FileStreamOptions
    {
        Options = FileOptions.WriteThrough
    });

// - OR -
var file = new FileStream(
    "1.txt", 
    FileMode.Create, 
    FileAccess.Write, 
    FileShare.None, 
    4096, 
    FileOptions.WriteThrough)