2

I have an app that keeps a database of files located on the user's machine or perhaps on networked volumes that may or may not be online. This database can potentially be several thousand files located in different folders. What is the best way to monitor them to receive notification when a file's name is changed, or it moves or is deleted?

I have used FSEvents before for a single directory but I am guessing that it does not scale well to a few thousand individual files. What about using kqueues?

I might be able to try to maintain a dynamic list of folders trying to encompass all the files with as few folders as possible, but this means reading though the full list and trying to figure out common ancestors etc.

Thoughts or suggestions?

Trygve
  • 1,317
  • 10
  • 27
  • What performance measurements lead you to the conclusion that FSEvents don't scale? – cacau Nov 28 '14 at 07:07
  • I would not be worried if I were adding a few directories with collectively a few thousand files, but adding a few thousand files individually seems like a lot of overhead. I don't want to spend the time and effort to build this if it is clearly going down the wrong road. – Trygve Dec 02 '14 at 02:43
  • Would be interesting to know how the Spotlight engine monitors the file system. I'd be surprised if they use some other/private API. – cacau Dec 03 '14 at 13:41

1 Answers1

0

From Apple's docs:

If you are monitoring a large hierarchy of content, you should use file system events instead, however, because kernel queues are somewhat more complex than kernel events, and can be more resource intensive because of the additional user-kernel communication involved.

https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/KernelQueues.html#//apple_ref/doc/uid/TP40005289-CH5-SW2

cacau
  • 3,606
  • 3
  • 21
  • 42
  • 1
    When trying to add 4,000 files (and thus 4,000) fsevent watchers, I start getting errors in the Console: fseventsd: implementation_added_client : failed to add client for path /Volumes/path/to/one/of/my/files followed by: fseventsd: implementation_register_rpc: implementation_added_client() => 22 – Trygve Dec 20 '14 at 16:45