2

Both iCloud Drive and OneDrive for Mac support remote files (or "Files On-Demand" as OneDrive calls it).

  • These files appear in Finder and seem to behave as real files (with odd Quick Look behavior).
  • The Inspector shows their true size plus (4 KB on disk) or similar.

Also:

  • Terminal does not seem to understand iCloud remote size:
: ~/Library/Mobile Documents/com~apple~Preview/Documents
$ ls -lh
total 24
-rw-r--r--@ 1 REDACTED  staff   157B Apr 28  2014 .2AM.png.icloud
-rw-r--r--@ 1 REDACTED  staff   6.0K Sep 30  2018 .DS_Store
  • But it understands OneDrive's (the file shows 0 bytes on disk in Finder):
: ~/OneDrive/Pictures/Drawings
$ ls -lh
total 24
-rwxrwxrwx@  1 REDACTED  staff    11M Apr  5  2019 MMM.kra*

Screenshot of iCloud Drive with cloud icons and "4kb on disk" in Inspector

Disappointingly, I cannot find any information about these files online. The official Apple File System Reference doesn't seem to show any clear endpoints for this.

  • How do they work?
  • Is this a public macOS API or feature of APFS? Is this dependent on APFS?
citelao
  • 4,898
  • 2
  • 22
  • 36

2 Answers2

1

It looks like they are using sparse files (a feature of APFS) to show the size without using up space.

In order to populate the file on-demand, they are using extended file attributes.

Try this command on your files to see more info: ls -l@

I haven't seen any documentation on how to use this anywhere, but apparently there is an attribute you can set to get a callback when another application opens the file. This works in conjunction with the com.apple.fileutil kext, which OneDrive uses. Microsoft must have special access.

j_6582
  • 161
  • 9
  • While you've already found most of it, see https://developer.apple.com/forums/thread/122351?answerId=650166022#650166022 for some additional notes including Dropbox's use of `vfs.fsplaceholder` as well. You can display the contents of the `ls -l@` attributes using `xattr -l some_placeholder_file.example.txt`. – natevw Jun 02 '21 at 23:20
0

It's filecoordinationd which handles all this magic. The files are marked as dateless placeholders, and filecoordinationd is able to "materialize" them on access by retrieving them from iCloud (as well as handle the reverse direction when they are saved or "evicted".

There's an Apple utility in /usr/bin/fileproviderctl which can be used to explore this further. (e.g. fileproviderctl materialize..)

Sparse files are something entirely different: They're for zero-filled files which can then take up far less space as their zero-ed blocks take no space.

Reference: NewOSXBook.com, Volume II.

Technologeeks
  • 7,674
  • 25
  • 36
  • Wow, that is fiddly! If you're the author of that book, cool! Seems like something to look into :D. Someone dumped the [manpage for fileproviderctl](https://keith.github.io/xcode-man-pages/fileproviderctl.1.html), but there's little else publicly available. – citelao Sep 16 '20 at 17:42
  • That's because it's an Apple internal utility that likely shouldn't have been left out. But there's nothing special about "dumping the man page" when you can simply "man fileproviderctl" on macOS since they left the man page too. There's an experiment in the book demonstrating how to use the tool. You can follow with dtrace to see the vfs resolve_* calls that occur in kernel – Technologeeks Sep 17 '20 at 12:50
  • "It's filecoordinationd which handles all this magic." I'm not sure this is correct? See e.g. https://developer.apple.com/forums/thread/122351?answerId=650166022#650166022 — there appears to be private `fsplaceholder` and/or `fileutil` kernel extensions which Dropbox and OneDrive leveraged prior to macOS 11. – natevw Jun 02 '21 at 23:14
  • kernel extensions indeed handle fs stuff at kernel level, but filecoordinationd is the orchestrator in user mode. – Technologeeks Jun 09 '21 at 23:09