File system watcher for Mac OS X

Question

Currently, we use QFileSystemWatcher, belonging to Qt. Due to the limited support in Mac OS X, it can only notify us two events: directory changed, or file changed.

However, the latter event (file changed) is triggered multiple times when its size is slightly larger and the writing to disk takes slightly longer time.

Our workaround is to set up a timer to check the file in 1 second. If more signals about the file come before timer expires, we reset the timer.

Is there a way to get the notification when the file is written to the disk (finished writing)? There is no need to limit to Qt, any library could do.

We are aware of the kqueue monitoring method, but that's too low level and we don't want to do that for each file, as we are monitoring a large portion of the file system..

QFileWatcher is sending fileChange signal constantly while file being written to the disk or just one time when it starts writing? — ixSci, Mar 04 '13 at 05:16
@ixSci it's sending sigals constantly; signals were dup'ed even for small files. — zzk, Mar 04 '13 at 05:26

score 2 · Answer 1 · answered Mar 05 '13 at 09:40

I have the same problem in a project and finally I decide to implement a native watcher. It's pretty easy:

In the .h:

class OSXWatcher : public Watcher
{
public:

    OSXWatcher(const QString& strDirectory);
    virtual ~OSXWatcher();

    virtual bool Start();
    virtual bool Stop();

private:

    /**
     * Callback function of the OS X FSEvent API.
     */
    static void fileSystemEventCallback(ConstFSEventStreamRef streamRef, void *clientCallBackInfo, size_t numEvents, void *eventPaths, const FSEventStreamEventFlags eventFlags[], const FSEventStreamEventId eventIds[]);

    FSEventStreamRef stream;
};

The .cpp:

bool OSXWatcher::Start()
{
    CFStringRef pathToWatchCF = CFStringCreateWithCString(NULL, this->dirToWatch.toUtf8().constData(), kCFStringEncodingUTF8);
    CFArrayRef pathsToWatch = CFArrayCreate(NULL, (const void **)&pathToWatchCF, 1, NULL);

    FSEventStreamContext context;
    context.version = 0;
    context.info = this;
    context.retain = NULL;
    context.release = NULL;
    context.copyDescription = NULL;

    stream = FSEventStreamCreate(NULL, &OSXWatcher::fileSystemEventCallback, &context, pathsToWatch, kFSEventStreamEventIdSinceNow, 3.0, kFSEventStreamCreateFlagFileEvents);
    FSEventStreamScheduleWithRunLoop(stream, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode);
    FSEventStreamStart(stream);

    CFRelease(pathToWatchCF);

    // Read the folder content to protect any unprotected or pending file
    ReadFolderContent();
}

bool OSXWatcher::Stop()
{
    FSEventStreamStop(stream);
    FSEventStreamInvalidate(stream);
    FSEventStreamRelease(stream);
}

void OSXWatcher::fileSystemEventCallback(ConstFSEventStreamRef /*streamRef*/, void *clientCallBackInfo, size_t numEvents, void *eventPaths, const FSEventStreamEventFlags eventFlags[], const FSEventStreamEventId eventIds[])
{
    char **paths = (char **)eventPaths;

    for (size_t i=0; i<numEvents; i++) {
        // When a file is created we receive first a kFSEventStreamEventFlagItemCreated and second a (kFSEventStreamEventFlagItemCreated & kFSEventStreamEventFlagItemModified)
        // when the file is finally copied. Catch this second event.
        if (eventFlags[i] & kFSEventStreamEventFlagItemCreated
                && eventFlags[i] & kFSEventStreamEventFlagItemModified
                && !(eventFlags[i] & kFSEventStreamEventFlagItemIsDir)
                && !(eventFlags[i] & kFSEventStreamEventFlagItemIsSymlink)
                && !(eventFlags[i] & kFSEventStreamEventFlagItemFinderInfoMod)) {

            OSXWatcher *watcher = (OSXWatcher *)clientCallBackInfo;
            if (watcher->FileValidator(paths[i]))
                emit watcher->yourSignalHere();
        }
    }
}

score 1 · Answer 2 · answered Mar 04 '13 at 05:35

I have the same problem but with folder. When you copy many files to the folder too many singals are emitted but I need only one. So I have the following solution:

void folderChanged(const QString& folder)
{
    m_pTimerForChanges->start();
}

folderChanged is a slot for directoryChanged() signal. And timer has another connection for timeout, so when time is out then processing should be done. Timer has 1s interval. Idea behind it is that folder should not be updated more frequent than interval I have and if it sends signals more frequently than I need then I don't need to process them immediately. Rather I restart timer every time the signal is emmited and with all of it I have the only one processing of changes. I think you can apply the same approach.

Another approach which may work for you also is to check file modification date in your processing and if its current modification date is within some epsilon(small interval) with your last modification date then you have repeating signal and should not react on it. Store this modification date and proceed.

This is basically similar approach as we did right now. Basically when ever we receive a signal for a file, we set a timer to check for it 1 sec later. If another signal comes, we reset the timer. So we also process the change only once. But the thing we don't like is the "arbitrary" time value and "unnecessary" delay caused by the timer.. although it may not be that important.. — zzk, Mar 04 '13 at 06:33
I believe Qt is using some system specific way to get notification about any changes. Hence you have to use some approximate approach anyway since I doubt there is some more precise way to do it even with plain system API. Also I'd recommend to try the method with modification date comparison. I think it should work for file changes(e.g. if file was changed within 500ms then it is part of one whole change not a different one). — ixSci, Mar 04 '13 at 06:41

File system watcher for Mac OS X

2 Answers2