18

I've had 3 reports now of user's machines crashing while using my software.. the crashes are not related to my program but when they restart the config files my program writes are all corrupt.

There is nothing special to how the files are being written, simply creating a Json representation and dumping it to disk using File.WriteAllText()

// save our contents to the disk
string json = JsonConvert.SerializeObject(objectInfo, Formatting.Indented);

// write the contents
File.WriteAllText(path, json);

I've had a user send me one of the files and the length looks about right (~3kb) but the contents are all 0x00.

According to the post below File.WriteAllText should close the file handle, flushing any unwritten contents to the disk:

In my C# code does the computer wait until output is complete before moving on?

BUT, as pointed out by Alberto in the comments:

System.IO.File.WriteAllText when completes, will flush all the text to the filesystem cache, then, it will be lazily written to the drive.

So I presume what is happening here is that the file is being cleared and initialized with 0x00 but the data is not yet written when the system crashes.

I was thinking of maybe using some sort of temp file so the process would be like this:

  1. Write new contents to temp file
  2. Delete original file
  3. Rename temp file to original

I don't think that will solve the problem as I presume Windows will just move the file even though the IO is still pending.

Is there any way I can force the machine to dump that data to disk instead of it deciding when to do it or perhaps a better way to update a file?

UPDATE:

Based on suggestions by @usr, @mikez and @llya luzyanin I've created a new WriteAllText function that performs the write using the following logic:

  1. Create a temp file with the new contents using the FileOptions.WriteThrough flag
  2. Writes the data to disk (won't return until the write has completed)
  3. File.Replace to copy the contents of the new temp file to the real file, making a backup

With that logic, if the final file fails to load, my code an check for a backup file and load that instead

Here is the code:

public static void WriteAllTextWithBackup(string path, string contents)
{
    // generate a temp filename
    var tempPath = Path.GetTempFileName();

    // create the backup name
    var backup = path + ".backup";

    // delete any existing backups
    if (File.Exists(backup))
        File.Delete(backup);

    // get the bytes
    var data = Encoding.UTF8.GetBytes(contents);

    // write the data to a temp file
    using (var tempFile = File.Create(tempPath, 4096, FileOptions.WriteThrough))
        tempFile.Write(data, 0, data.Length);

    // replace the contents
    File.Replace(tempPath, path, backup);
}
Community
  • 1
  • 1
antfx
  • 3,205
  • 3
  • 35
  • 45
  • 1
    I honestly think your time would be better spent, in this case, determining what's causing the systems to crash. – user2366842 Aug 18 '14 at 15:15
  • 2
    they are just random crashes on people's machines that have nothing to do with my system (one was reported as being a driver issues I think). Anyway the point is that as the software developer my users are asking me to make sure my program doesn't loose its configs if/when they get a BSOD or whatever – antfx Aug 18 '14 at 15:17
  • 1
    Have you tried using `File.Create` method instead of `File.WriteAllText`? It has useful overload - http://msdn.microsoft.com/en-us/library/ms143360(v=vs.110).aspx, it takes FileOptions parameter, which can be set to `WriteThrough` - "Indicates that the system should write through any intermediate cache and go directly to disk." – Ilya Luzyanin Aug 18 '14 at 15:21
  • 1
    You might look at creating the `FileStream` manually and using the `FileOptions.WriteThrough` option. [This](http://stackoverflow.com/questions/5916673/how-to-do-non-cached-file-writes-in-c-sharp-winform-app) question shows how to disable all buffering, which can only be done through PInvoke. – Mike Zboray Aug 18 '14 at 15:21
  • the FileOptions.WriteThrough option looks like exactly what I need – antfx Aug 18 '14 at 15:25
  • @miked only if you can guarantee that the data is at most one cluster in size. After that, writes are not atomic. – usr Aug 18 '14 at 15:25
  • @usr, can you clarify that? looking at MS docs it says "write call doesn't return until the data is written to the file.", implying that any data written goes in without any caching http://support.microsoft.com/kb/99794/en-gb – antfx Aug 18 '14 at 15:29
  • 1
    @miked if you write 1GB using WriteThrough I think it is intuitive to believe that the data can be partially written. When the call returns the data is stable, but before that you can have a mixture of old and new data. – usr Aug 18 '14 at 15:29
  • I bellieve the temp-file is way to go. You could also validate the tempfile before moving, or backup the old config and let it resore if it fails to load – CSharpie Aug 18 '14 at 15:41
  • @CSharpie you can't validate the temp file because the OS will give you cached data. You also cannot backup because you can't be sure the backup is on disk. A safe flushing mechanism is required. – usr Aug 18 '14 at 15:43
  • @antfx Did you "fix fixed"? I have the same problem! – Francesco Bonizzi Aug 06 '20 at 13:36

1 Answers1

15

You can use FileStream.Flush to force the data to disk. Write to a temp file and use File.Replace to atomically replace the target file.

I believe this is guaranteed to work. File systems give weak guarantees. These guarantees are hardly ever documented and they are complex.

Alternatively, you can use Transactional NTFS if available. It is available for .NET.

FileOptions.WriteThrough can replace Flush but you still need the temp file if your data can exceed a single cluster in size.

usr
  • 168,620
  • 35
  • 240
  • 369
  • That only affects the temp file. The data is simply lost, then. The old data stays. – usr Aug 18 '14 at 15:20
  • Makes sense. I suppose in this case, losing the new data is probably better than a corrupted config. – user2366842 Aug 18 '14 at 15:28
  • yes, having a copy of the old data is fine.. the user looses a bit of info but not much – antfx Aug 18 '14 at 15:30
  • @miked loosing data is an unsolvable problem because the machine might bluescreen right before you start writing. Crash-consistency is solvable, though. – usr Aug 18 '14 at 15:31
  • I've updated my question with the new write function.. I think the logic is sound? – antfx Aug 18 '14 at 16:01