FileSystemWatcher changed event (for "LastWrite") is unreliable

Question

I am trying to get a notification when a file is updated on disk. I am interested in getting this notifications as soon as a flush occurs, yet, it seems that the FileSystemWatcher would only send an event when the stream opens or closes.

In the code below I am writing to a file and repeatedly flush the buffer to the disk. Yet, the FileSystemWatcher only notified me once in the beginning of the writes and once when they end.

Is there another way to get these notifications? Or should I use polling for that?

The code:

class Program
{
    static void Main(string[] args)
    {
        FileSystemWatcher watcher = new FileSystemWatcher(Environment.CurrentDirectory, "Test.txt");
        watcher.Changed += watcher_Changed;
        watcher.EnableRaisingEvents = true;

        using(TextWriter writer = new StreamWriter("Test.txt"))
        {
            WriteData(writer);
            WriteData(writer);
            WriteData(writer);
            WriteData(writer);
        }

        Thread.Sleep(10000);
    }

    private static void WriteData(TextWriter writer)
    {
        writer.WriteLine("Hello!");
        writer.Flush();
        Console.WriteLine(DateTime.Now.ToString("T") + "] Wrote data");
        Thread.Sleep(3000);
    }

    static void watcher_Changed(object sender, FileSystemEventArgs e)
    {
        Console.WriteLine(DateTime.Now.ToString("T") + "] Watcher changed!");
    }
}

UPDATE 1

I've corrected the ToString function of the DateTime to show seconds. Here is the output with the code above:

11:37:47 AM] Watcher changed!
11:37:47 AM] Wrote data
11:37:50 AM] Wrote data
11:37:53 AM] Wrote data
11:37:56 AM] Wrote data
11:37:59 AM] Watcher changed!

Thanks!

Answer 1

It has nothing to do with the FileSystemWatcher. The watcher reacts to updates on the LastWrite attribute for the filesystem.

E.g. NTFS does not update the LastWrite on every write. The value is cached and only written when the stream is closed or at some other unspecified time. This document says

Timestamps are updated at various times and for various reasons. The only guarantee about a file timestamp is that the file time is correctly reflected when the handle that makes the change is closed. [...] The NTFS file system delays updates to the last access time for a file by up to 1 hour after the last access

I assume a similar caching applies for write

Answer 2

I think one of your issues here is that all your writes get executed before the event gets a slice of that cpu-time. MSDN states

The Changed event is raised when changes are made to the size, system attributes, last write time, last access time, or security permissions of a file or directory in the directory being monitored.

I did a test and inserted Sleeps after each call to WriteData(...), what i got was

09:32] Watcher changed!

09:32] Watcher changed!

09:32] -----> Wrote data

09:32] -----> Wrote data

09:32] -----> Wrote data

09:32] -----> Wrote data

09:32] Watcher changed!

09:32] Watcher changed!

I guess this kind of proves that the even is fired right after you call Flush(), it's just a question of when the event-handler gets executed (and I assume it groups events as well). I don't know the specific needs of your project, but I wouldn't poll. Seems like a waste since FileSystemWatcher does what you want it to do in my opinion.

Edit: Ok, I guess my brain wasn't quite ready yet for thinking when I posted this. Your conclusion that it fires when you open and close the stream seems more logical and right. I guess I was looking for "prove" that it fires when you call flush and therefore found it - somehow.

Update I just had a poke at the USN-Journal and it seems that wont get you what you want either as it writes the record only when the file is closed. -> http://msdn.microsoft.com/en-us/library/aa363803(VS.85).aspx I also found a USN-Viewer in C# and http://social.msdn.microsoft.com/Forums/en/csharpgeneral/thread/c1550294-d121-4511-ac32-31551497f64e might be interesting to read as well.

I also ran DiskMon to see if it gets the changes in realtime. It doesn't, but I don't know if that's intentional or not. However, the problem with both is that they require admin-rights to run. So I guess you are stuck with FileSystemWatcher. (What is it you need the updates for anyway, it's not like you can read the file while it's open/locked by another program.)

ps: I just noticed you are a dev of BugAid, which I was made aware of it only recently - it looks awesome :)

Answer 3

I solved this by using a timer:

private Timer _checkTimer;
private DateTime _lastSaved;

Before initializing the timer, I get the last write time of the file:

_lastSaved = File.GetLastWriteTime(_watchedFile);
_checkTimer = new Timer() { Interval = 1000, AutoReset = true };
_checkTimer.Elapsed += CheckTimer_Elapsed;
_checkTimer.Start();

Then on each elapsed event I check if the last write time is newer than the one that I knew to be the last write time:

private void CheckTimer_Elapsed(object sender, ElapsedEventArgs e)
{
    DateTime lastWriteTime = File.GetLastWriteTime(_watchedFile);
    if (lastWriteTime > _lastSaved)
    {
       // Stop the timer to avoid concurrency problems
       _syncTimer.Stop();

       // Do here your processing

       // Now, this is the new last known write time
       _lastSaved = File.GetLastWriteTime(_watchedFile);

       // Start again the timer
       _syncTimer.Start();
    }
}

This is how I personally solved this problem that I also had with FileSystemWatcher. This method is not as expensive as reading the file content or computing the file hash every second to check if there is a difference.

Answer 4

On windows (10) and NTFS, if you are in control of the writing app you can force the update of write timestamp and it will be picked up the FileSystemWatcher.

In C# (and .NET) you can use FileStream.Flush(true)