Develop a userspace read-only disk driver for macOS

Question

I have a special disk image format (compressed and with extra information such as bad blocks) and would like to make that available as a block device (e.g. under /dev/rdisk) in macOS.

I am looking for pointers on how to write a read-only block level userspace driver.

I believe I need to do this with IOKit, but the docs are quite sparse in that area, and provide no sample code for this as far as I can tell.

I've had a look at Amit Singh's Mac OS Internals, but that only goes about filerting existing blocks that get routed through the added driver. But I need to read the data from a separate file, i.e. I need to use the file system, and I also want to do this in a userspace app if possible because making a kext is both hard to debug and is getting deprecated now, too.

Ideally, this should work on current macOS versions, including 10.15, but I'd also be happy with a solution that only works on older macOS versions, starting at 10.6.

Maybe I am still misunderstanding some things here. I was under the impression that it was possible even before 10.15 to write userspace IOKit drivers. But maybe that's not possible with Block Storage devices? Any clarifications would be welcome. I feel quite lost.

Update June 22, 2020

I also asked the question on the OSXFUSE support forum, and got a negative reply there.

Answer 1

Maybe I am still misunderstanding some things here. I was under the impression that it was possible even before 10.15 to write userspace IOKit drivers.

This is a true statement, but it does not mean it's possible to write all types of drivers in userspace, even with 10.15. The confusion likely stems from the fact that "IOKit" is used to mean subtly different things in different contexts:

• Fundamentally, it is the (class and inheritance) OOP-based driver stack technology which is built into the XNU kernel (macOS, iOS, etc.), written in C++. Its core data structure is the I/O Kit registry, which is a directed graph of objects representing devices, device (sub-)functions, virtual device, service, drivers, user space client connections, etc. (most of it has properties of it, but nodes can technically have multiple "parent" nodes, and some actually do so on your typical Mac system)
• The IOKit user space libraries. (a.k.a. IOKit.framework a.k.a. IOKitLib + various IOCFPlugins) These provide a view of the kernel's I/O Registry to user space programs, and allow them to perform certain interactions with IOKit objects. This is essentially limited to:
  • Iterating over the registry graph and matching services based on matching rules. Both return handles to specific I/O Registry entries to the user space program.
  • Getting properties on I/O Registry entries, and if the object explicitly supports it, setting (some) properties.
  • Creating a user client connection. This is the only way to create an entry in the I/O Registry from regular user space, and these are always IOUserClient subclasses, and only if the parent object specfically permits it.
• The IOKit view available to DriverKit based System Extension processes (new in 10.15), which allows instantiation of entirely user space backed registry entry objects, for subclassing a (bounded) selection of super class types.

To answer the question of your specific case:

• To implement a driver for a block storage device which will appear to the system as any other, you need to subclass IOStorage or one of its subclasses (typically IOBlockStorageDevice).
• IOKit.framework allows you to create user space drivers which will only be used by that user space process or other user space processes. It is therefore not possible to use it to create drivers that will be used by some part of the kernel. For example, it is not possible to create a block device driver which will be used by the kernel's VFS system.
• DriverKit as of macOS 10.15.4 SDK does not allow dext-backed subclasses of any of the storage stack's classes. So a dext will not solve the problem on 10.15.
  Update: SCSIControllerDriverKit was added with DriverKit 20.4 (macOS 11.4), while BlockStorageDeviceDriverKit came along with DriverKit 21 (macOS 12). I believe these are intended to be used for PCI devices, but it may be possible to build a "virtual" block device with them too. (You may struggle to obtain the necessary DriverKit entitlements for this purpose, and will likely have an even harder time publishing such a driver to the App Store. Moreover, SCSIControllerDriverKit only started providing access to transfer data buffers in the driver from DriverKit 21 onwards, so there's no API availability advantage from the point of view of writing a virtual device driver.)
• The only way to do what you're trying to do, as far as I'm aware, is to have a 2-part driver: a kext implementing an IOBlockStorageDevice subclass, as well as a service which offers a user client interface for some user space process to connect to for creating instances of block devices, and which will handle the I/O requests for the device. This is a sort of analog to the FUSE mechanism which works at the VFS layer instead of the block layer.
  This is exactly how macOS's built-in disk image mounting system works. Unfortunately, neither the kext nor user space parts of this are open source, documented, or a public API. So unless you reverse engineer it and find a boundary which doesn't enforce the requirement of an Apple code signature, you basically have to reimplement this.

I'm not currently aware of an existing open source system that does this, but I haven't looked very hard. Especially for read-only access, it shouldn't be terribly hard to do if you know what you're doing, but it comes with the usual downsides of developing and distributing kexts.

Note that you could also use FUSE to open your disk image if the block device it represents hosts a file system you wish to mount, and a FUSE implementation for that file system exists.