1

A linux block device driver allocates a struct gendisk and any associated metdata, and provides a queue or some implementation of the submit_bio function to handle incoming I/O requests from the operating system.

When a disk is being removed, it calls:

del_gendisk(brd->brd_disk);
blk_cleanup_disk(brd->brd_disk);
...
kfree(brd);

see for example, the brd_cleanup routine for the ramdisk driver.

How does the driver know that there is no longer any code running (in submit_bio or in the bio_end_io functions) that might access the gendisk or metadata struct (free'd by kfree(brd) in the example above).

I see that del_gendisk et. al. freeze the queue and don't allow any more I/O, but from perusing the source it seems like this just stop new submissions to submit_bio. It does not track anything past the actual submit_bio call.

The _disk cleanup functions probably block until all open file handles to the device or its partitions are closed. Where is that code?

Reinstate Monica
  • 588
  • 7
  • 21

1 Answers1

0

blkdev_close is the higher level call which waits for all outstanding I/O. It calls __sync_blockdev to get everything written out. https://elixir.bootlin.com/linux/v4.20.17/source/fs/block_dev.c#L457

stark
  • 12,615
  • 3
  • 33
  • 50
  • That seems like what is called when a process closes a previously opened block device. I can't see where the device cleanup code call `.release` -> `blkdev_close` on all open files. Maybe I'm missing something? – Reinstate Monica Aug 13 '23 at 19:14
  • Other way around. You can't remove a driver when the reference count on the device is non-zero. blkdev_get (open) increments the reference count. If nothing has the device, then you are free to remove the driver. – stark Aug 13 '23 at 19:56
  • I think what you mean is that the `fops->owner` member is set to the module of the driver, and `blkdev_get` does a `try_module_get` on that, preventing the module from being unloaded (so, for example, `brd_cleanup` will never run until all handles are closed). Does this mean that drivers that allow removal of a specific device without unloading the module, must keep track of inflight i/o themselves; or is there some other code in `del_gendisk` that waits synchronously until some refcount reaches 0 - where is that code? – Reinstate Monica Aug 13 '23 at 20:38
  • blkdev_put calls release when bd_openers gets to 0. – stark Aug 13 '23 at 21:27
  • But a regular `blkdev_put` would be async. If `del_gendisk` etc. would call `blkdev_put` while the device is open, it would return immediately, leaving the handle open (only to be released when the handle is closed). When `kfree` is called on the disk, the open handle might still access the disk struct. – Reinstate Monica Aug 13 '23 at 22:05