C# - Predict file system events on folder delete

Question

This is more a question about what's the best practice in implementing this.

I have a FileSystemWatcher which should inform me about User's changes on files and folders. Also subdirs are watched. In the same directory my program does also sometimes changes. I don't want the FileSystemWatcher to detect events on these program changes.

My first implementation was a list where I can add expected events. When I get a file system event I check the list and ignore it if it's there. This doesn't sound very robust but it seemed to work.

Now I detected the real problem:
D: is watched by the FileSystemWatcher.
I have two folders like that: D:\folder1\folder2
Now I want to delete folder1 (with folder2 in it) with my application. So I put D:\folder1 in my delete list. Then I call something like Directory.Delete(@"D:\folder1", true) . Now I notice that folder1 can't be deleted (why ever) over an exception. I remove the delete entry from my list but folder2 was already deleted and I get his FileSystemEvent. So I get a FileSystem event for D:\folder1\folder2. My program thinks now the user has deleted this folder and is doing the wrong things.

I had some ideas now:

1.) recursively delete the folder by deleting every file and every folder by it's own. With this I get for every subfolder and file an own list entry. I already implemented it but it is very very very slow.

2.) Maybe there is a better way to have clever filters in the FileSystemWatcher to make my list obsolete?

3.) Maybe it is possible to only delete a directory tree if it's possible to delete everything. So if it fails I still have everything and if not everything is deleted. This seems to be the most elegant solution for me but no idea if this is even possible?

4.) Is it possible to exclusively lock all files and folders by my software? If this this went ok it should be possible delete everything with one delete command or somehow like this?

I'm also open for other additional solutions.

Edit 1 to make it more clear:

I only want to "see" user actions on a folder. If I manipulate things from my program in here I don't want to see this events.

With my implementation I get events for subfolders if a folder is locked and can't be deleted.

It's not so easy to explain in english because I'm no english native speaker ;).

Edit 2:

5.) Maybe it is possible to filter in FileSystemWatcher all events from a defined process?

Answer 1

I've done exactly this sort of thing recently; the trick is to have your 'list' recognise that where there's a folder name in the list, also discard any events for anything within that folder if it's expecting a delete event, and only remove it from your prediction list if it's the folder itself.

I should warn you though, you're likely to encounter problems with the FileSystemWatchers buffer becoming full if too many events happen in rapid succession; if this does, it'll fire an Error event, and fail to notify you of a whole bunch of events. If your prediction list removes items as it receives the event, you run the risk of ignoring future events just because the event you intended to ignore was never received due to a buffer overflow. It may also get very large, as items are never being removed from the list.

I've yet to find a reliably way of doing this, although you can set the size of the FileSystemWatchers buffer to maximum to mitigate it to a degree.

EDIT: Also very important: The events that come back do so on a different thread (unless your handler is on an object that implements ISynchronizeInvoke, such as a Control or Form, and you set the SynchronizingObject to your Control/Form. This means you need to be very careful how you maintain your list, taking into account potential race conditions. I'm still struggling with this one; my code flushes the prediction list on receiving an error event, but by the time it handles the event, other change events since the error have already been fired and handled, and it flushes things it shouldn't.

Answer 2

Regarding the buffer overflow. The best way to solve it is to do the work in response to an event on another thread. I do it like this

// Queue of changed paths.
private readonly Queue<string> mEventQueue = new Queue<string>();

// add this as handler for filesystemwatcher events
public void FileSystemEvent(object source, FileSystemEventArgs e) {
    lock (mEventQueue) {
        if (!mEventQueue.Contains(e.FullPath)) {
            mEventQueue.Enqueue(e.FullPath);
            Monitor.Pulse(mEventQueue);
        }
    }
}

// start this on another thread
public void WatchLoop() {
    string path;
    while (true) {
        lock (mEventQueue) {
            while (mEventQueue.Count == 0)
                Monitor.Wait(mEventQueue);
            path = mEventQueue.Dequeue();
            if (path == null)
               break;
        }
        // do whatever you want to do
     }
}

That way I will never miss an event

Answer 3

Ok here is my solution of the problem:

Just for delete command:

I implemented 2 lists, one for file deletes and one for folder deletes.

file list entrys have no timeout. If I delete a file I create a list entry and if I get the expected delete event I remove the entry as before.

folder list entrys have no timeout after create. I can command them manually to time out after a second by a special method. If I delete a folder I add a deleteFolder list entry. Every delete event from this folder or file or subfolder or subfolderfile is ignored because of this delete folder entry. After the delete is finished I arm the timeout for the deleteFolder entry. If the delete throws an exception I do the same. So I expect to get all events after a second. So I'm ignoring all events if the delete command works or not. After this the deleteFolder list entry gets deleted.

Limitations: 1. all delete events have to come in one second after the delete happened. 2. it is not allowed to delete a folder like this:
delete folder (completed)
create folder again
wait less then 1 second
delete folder again (timeout has not finished of the delete folder list entry)

Hope this helps somebody even it is very complicated ^^