0

I am designing a packaged-based NSDocument file format. The document will contain lots of directories and files, so I want to save a cache, either as part of the document, or in the application's Caches directory.

The cache would be a simple plist array.

I want to design this so the document can be synced via Dropbox or iCloud. So I'm up against one of the hard problems... cache invalidation.

If I store the cache in the document package, if the document is edited on two machines at once, either one might overwrite the cache of the other, or they might need conflict resolution.

If I store the cache in the external Caches directory, perhaps named based on the document name or file identifier, I need some way to detect that it's stale if the document was updated elsewhere (via sync or even just a newer version copied from an email).

An external cache also has other issues, e.g. I thought about storing a UUID in the document and using that as the cache filename, but if the document is duplicated, the cache would be used by two documents, which is obviously wrong as soon as one is edited.

It seems to me that the safest place for it is in the document package, but then it comes back to a possible conflict.

I also wondered how to detect changes. One overly-complex idea was to write a UUID-named file to a folder within the document every time it is saved, then if the number of files there changes, rebuild the cache. (A simpler approach would be a change count in the document, but again that could result in a conflict if two machines increment the count and save at the same time, bearing in mind there could be a syncing delay.)

It seems to me that safely syncing a document package isn't too an uncommon problem, so I'm hoping someone can offer some best practices recommendations?

Dejal
  • 813
  • 6
  • 18

2 Answers2

1

With Dropbox you'll run into the problem that there is no way to change a document package atomically. That is, if saving your document involves changing N files, and then saving again involves changing M different files, Dropbox will sync those N+M changed files in whatever order it wants and you can't tell where one document level save ends and the next began or whether you are currently seeing a complete save or some intermediate value where you have some updated files but haven't seen others yet.

iCloud knows about document packages, but at least right now, you'll occasionally see spurious conflicts or a change "going up into the cloud" and just seemingly get lost and never come back down again. Debugging these sorts of situations is very, very difficult.

gregomni
  • 941
  • 5
  • 3
  • Thanks. Sync is certainly hard... you'd think in 2013 it would work better. I guess for Dropbox I'll just have to re-cache after each change. Hopefully NSFilePresenter will help track what was changed while running. – Dejal Mar 18 '13 at 19:36
0

Do not store the cache inside the document package. You should store it in the Caches (NSCachesDirectory) directory.

To safely associate the cache with a document, you can create a cache file name that is based on the file system path of your document. For example by creating suffix that is a hash of the path. This means that if the document is moved (outside of your application), the cache will have to be recreated - but that seems like a reasonable trade-off.

Xcode caches project related files a directory DerivedData/<project name>-<identifier>. If you look into that directory, you'll find a info.plist file that contains the path of the Xcode project - presumably used by Xcode to verify that the cache belongs to the project. Curiously, someone reversed engineered the function that Xcode uses to create the identifier, and posted in on this blog: https://pewpewthespells.com/blog/xcode_deriveddata_hashes.html.

guru_meditator
  • 549
  • 6
  • 11