23

I understand that .NET FileStream's Flush method only writes the current buffer to disk, but dependent on Windows' disk driver and the hard disk firmware this is no guarantee that the data is actually physically written to disk.

Is there a .NET or Win32 method that can give me this guarantee? So if there is power loss one nanosecond after the call to this method comes back, I can still be sure that everything is OK?

Stefan Schultze
  • 9,240
  • 6
  • 35
  • 42

7 Answers7

17

Stefan S. said:

I understand that .NET FileStream's Flush method only writes the current buffer to disk

No, .NET FileStream's Flush only writes the .NET buffers to the OS cache, it does not flush the OS cache to disk. Sadly the MSDN doc on this class doesn't say that. For .NET < 4.0, you'll have to call Flush + Win32's FlushFilebuffers:

using System.Runtime.InteropServices;
. . .

// start of class:
[DllImport("kernel32", SetLastError=true)]
private static extern bool FlushFileBuffers(IntPtr handle);
. . .

stream.Flush();     // Flush .NET buffers to OS file cache.
#pragma warning disable 618,612 // disable stream.Handle deprecation warning.
if (!FlushFileBuffers(stream.Handle))   // Flush OS file cache to disk.
#pragma warning restore 618,612
{
  Int32 err = Marshal.GetLastWin32Error();
  throw new Win32Exception(err, "Win32 FlushFileBuffers returned error for " + stream.Name);
}

For .NET 4.0, you can instead use the new flush(true) method. 11/09/2012 update: MS bug report here says it's broken, then fixed, but doesn't say what version or service pack it was fixed in! Sounds like bug was if internal .NET FileStream buffer is empty, the Flush(true) did nothing??

jimvfr
  • 353
  • 5
  • 10
9

Under Windows, look at FlushFileBuffers (Win32 API).

Pavel Radzivilovsky
  • 18,794
  • 5
  • 57
  • 67
  • Thank you :) I have quickly done a performance test and FileStream.Flush() is way too fast to be true. FlushFileBuffers on FileStream's SafeFileHandle is as slow as I would expect it (100 times slower than Flush() in my test) – Stefan Schultze Dec 20 '08 at 14:22
  • 1
    I found that calling FlushFileBuffers can cause an exception (http://stackoverflow.com/q/9195807/4540). Under .NET 4, it's easier and safer to just call FileStream.Flush(true) as @jimvfr suggests (http://stackoverflow.com/a/3992428/4540). – Eric Feb 08 '12 at 15:36
  • The file data that's buffered in the file system cache to be written to disk. That data is normally lazily written, based on the position of the disk write head. Having a gigabyte of cached data is technically possible so it can take quite a while. If this is important to you then consider the FileOptions.WriteThrough option instead. – MSH Sep 21 '16 at 14:24
  • Is there a way to apply this logic to Registry as well? – bytecode77 Aug 28 '17 at 21:26
4

Well, you could close the file... that would probably do it. In reality, with HAL abstraction, virtualization, and disk hardware now having more processing power and cache memory than computers did a few years ago, you're going to have to live with hoping the disk does its job.

The transactional file system never really materialized ;-p Of course, you could perhaps look at using a database as a back end, and use the transaction system of that?

Aside: note that not all streams even guarantee to Flush() - for example, GZipStream etc retain a working buffer of uncommitted data even after a flush - the only way to get it to flush everything is to Close() it.

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • Technically, with writing to a database, theres no guarantee that a power outage or other catastrophic failure won't lose the write or corrupt the database somehow, although its far more likely to survive than a simple filesystem write. – cletus Dec 20 '08 at 14:14
  • 1
    yes, but you can wrap the "mark this as done" and "here's the results" in the same transaction – Marc Gravell Dec 20 '08 at 14:32
  • @cletus If a database system doesn't guarantee that then it's either broken (at least I consider a DBMS that's not ACID broken) or running on a broken system (OS, hardware, whatever). – Paul Groke Mar 26 '19 at 23:31
3

I've noticed that the .NET 4 #Flush(true) doesn't actually write to the disk. We were having strange issues with corrupted data and I found this bug report on the MS site:

The details tab for the bug report has a test program you can run that will show the issue;

  1. Write a bunch of data to disk
  2. fs.Flush(true). This takes no time (much faster than can possibly written to the disk).
  3. Use the win32 API FlushFileBuffers. This takes a long time.

I'm changing over to the win32 FlushFileBuffers call...

Adi Lester
  • 24,731
  • 12
  • 95
  • 110
dave
  • 31
  • 1
  • fs.Flush(true) is working for me just fine. Windows 10 x64 creators update, .NET 4.5 – jjxtra Jun 12 '18 at 19:14
  • @jjxtra: Did you test with a network server and physically pulling the plug? – Joshua Oct 12 '18 at 20:17
  • @Joshua nope, only tested with local file system, so can't speak for that scenario – jjxtra Oct 12 '18 at 21:35
  • @jjxtra: It will appear to work unless you cause the disks to fail by unplugging them. You need the network server to *know* you triggered the race. – Joshua Oct 12 '18 at 21:37
0

The file data that's buffered in the file system cache to be written to disk. That data is normally lazily written, based on the position of the disk write head. Having a gigabyte of cached data is technically possible so it can take quite a while. If this is important to you then consider the FileOptions.WriteThrough option instead.

MSH
  • 193
  • 2
  • 16
  • 1
    This has been proven to be ineffective as well. The OS will still cache. – trevster344 Jan 10 '18 at 15:59
  • @trevster344, Do you have a source for that? – Matt Jun 13 '18 at 15:00
  • @Matt `FILE_FLAG_WRITE_THROUGH` used to be broken with SATA drives: https://disruptivesql.wordpress.com/2012/05/08/sata-and-write-through/ I don't know the current state of affairs, but I wouldn't trust it without further research and testing. – Paul Groke Mar 26 '19 at 23:27
0

There is a simple answer to flushing the content of the buffer to disk. After your WriteAllText function, open file, close it, and reset it

here is an example

My.Computer.FileSystem.WriteAllText(yourfilename, "hello", False, System.Text.Encoding.ASCII)
FileOpen(1, yourfilename, OpenMode.Input)
FileClose(1)
Reset()
Mickey2
  • 11
  • 2
-3

There's simply too many levels of abstraction to be absolutely sure that the data is written to the disc, right down to the hardware level.

Not brilliantly performant or foolproof, but how about re-opening the file once it is written in a seperate process and checking the size or contents?

Ayresome
  • 119
  • 2