0

I have question about device removal.

When we want to notify PnP manager that device has disappeared we call IoInvalidateDeviceRelations with BusRelations. After that OS will send IRP_MN_QUERY_DEVICE_RELATIONS request with BusRelations. In this request handler we will exclude device from array and will do another necessary job to "disconnect" it from bus, also we will set RemovePending flag in its device extension.

I don't understand how to deal with incoming IO requests to the device after it becomes "remove pending" and before OS sends IRP_MN_REMOVE_DEVICE request. Should we check RemovePending flag and return STATUS_DEVICE_DOES_NOT_EXIST or should we proceed as usual?

Now imagine that IRP_MN_REMOVE_DEVICE request has finally arrived. MSDN says that we must call IoReleaseRemoveLockAndWait that releases the current acquisition of remove lock, prevents subsequent acquisitions and waits while existing acquisitions are released. So it forces us to always acquire remove lock inside PnP request handler with IoAcquireRemoveLock; and release it with IoReleaseRemoveLockAndWait for IRP_MN_REMOVE_DEVICE or with IoReleaseRemoveLock for another minor codes.

I don't understand why we need to acquire remove lock inside PnP request handler? In my understanding we need to acquire remove lock only for pending irp and release it when such irp is completed. So Windows folks could provide us with IoWaitForExistingRemoveLocks routine instead of IoReleaseRemoveLockAndWait.

Sorry if it's kinda messy, I just can't figure it out. Thanks.

new_user
  • 3
  • 1

1 Answers1

1

After that OS will send IRP_MN_QUERY_DEVICE_RELATIONS request with BusRelations. In this request handler we will exclude device from array and will do another necessary job to "disconnect" it from bus, also we will set RemovePending flag in its device extension.

here need only exclude device from array and set RemovePending flag in its device extension. but do another necessary job to "disconnect" it from bus - need do only when you process IRP_MN_REMOVE_DEVICE (after device was not included in the bus driver's most recent response to an IRP_MN_QUERY_DEVICE_RELATIONS request for BusRelations - or in another words - when RemovePending flag in its device extension)

how to deal with incoming IO requests to the device after it becomes "remove pending" and before OS sends IRP_MN_REMOVE_DEVICE request. Should we check RemovePending flag and return STATUS_DEVICE_DOES_NOT_EXIST or should we proceed as usual?

i think possible both behavior - you can process it as usual and can return STATUS_DEVICE_DOES_NOT_EXIST too. and assume next situation - you get some IO request in concurrent with removal device process. when you check RemovePending flag - it yet not set. and you begin process request "as usual". but just after you check RemovePending flag inside IO request you can set it while handling IRP_MN_QUERY_DEVICE_RELATIONS request with BusRelations. and this situation direct related to sense using Remove Locks or Run-Down Protection.

we really can use Run-Down Protection instead Remove Locks almost in same way and exactly in same places and for same reason as Remove Locks. and i think Run-Down Protection api (more new compare remove locks) - better design and better for use (however different is minimal)

I don't understand why we need to acquire remove lock inside PnP request handler?

first of all note that about remove lock said only in Removing a Device in a Function Driver. you how i understand have not function, but bus driver- so Removing a Device in a Bus Driver more suitable for you. and in documentation for Removing a Device in a Function Driver exist serious error - advice first call IoReleaseRemoveLockAndWait at point 4 - before point 8 - Pass the IRP_MN_REMOVE_DEVICE request down to the next driver. but correct must be

driver should call IoReleaseRemoveLockAndWait after it passes the IRP_MN_REMOVE_DEVICE request to the next-lower driver, and before it releases memory, calls IoDetachDevice, or calls IoDeleteDevice.

this is correct and stated in Using Remove Locks and IoReleaseRemoveLockAndWait. funny that in old msdn versions was

call IoReleaseRemoveLockAndWait before it passes..

but now this is fixed. why after ? because next-lower driver can pending some IRPs (on which we call IoAcquireRemoveLock or ExAcquireRundownProtection) and complete it only when got IRP_MN_REMOVE_DEVICE and our driver call IoReleaseRemoveLock or ExReleaseRundownProtection only when this IRP will be completed. as result if call IoReleaseRemoveLockAndWait or ExWaitForRundownProtectionRelease before passes the remove IRP to the next-lower driver - we can wait here forever - next-lower driver can not complete some IRP (until not got remove request) and we not release remove lock or rundown protection.

so for what we need remove locks or rundown protection ? because we can got IRP_MN_REMOVE_DEVICE in concurrent with another IO requests. and this IO requests can use some resources on device. from another sized when we process IRP_MN_REMOVE_DEVICE we destroy this resources. what be if we will use some resource in IO request after he will be destroyed in IRP_MN_REMOVE_DEVICE ? think not need answer. for prevent this and exist removal locks or rundown protection. before use any resource (which will be destroyed in remove) need call IoAcquireRemoveLock or ExAcquireRundownProtection and use it only if ok status returned. after we finish use resource call IoReleaseRemoveLock or ExReleaseRundownProtection. and in IRP_MN_REMOVE_DEVICE we call IoReleaseRemoveLockAndWait or ExWaitForRundownProtectionRelease. after this call is returned - we can be sure that nobody use our resources and never will be used more (calls to IoAcquireRemoveLock or ExAcquireRundownProtection return error status (false)). at this point we can safe begin destroy resources: releases memory (etc), calls IoDetachDevice, IoDeleteDevice.

the pointer to the next-lower device - this is also resource, which we use in process IO request and destroy in IRP_MN_REMOVE_DEVICE (by call IoDetachDevice). really are correct call IofCallDriver(_nextDeviceObject, Irp); (while process some IO request) after call IoDetachDevice(_nextDeviceObject); (inside IRP_MN_REMOVE_DEVICE) ? because this removal locks (or i use yourself rundown-protection here ) always used in functions and filter drivers. for bus driver, where we usually have not pointer to the next-lower device (PDO not attached to another device, when FDO attached to PDO and filter always attached to something) - may be not need locks (or rundown protection at all). this is depend - are exist another resources - used and destroyed (on removal).

and for bus devices - usual situation when we got IRP_MN_REMOVE_DEVICE 2 time - first before device marked as RemovePending - so we go to point 4. and after device marked as RemovePending (so device was not included in the bus driver's most recent response to an IRP_MN_QUERY_DEVICE_RELATIONS request for BusRelations) we finally destroy resources and call IoDeleteDevice

RbMm
  • 31,280
  • 3
  • 35
  • 56
  • @RbMmThank you for the answer, it is very clearly. So the idea is to protect resources (device object, its extension, etc) from being freed while we are processing another request. The thing that I still dont understand: we store remove lock in the device extension, so to acquire it we need to access this extension. Where is the guarantee that by the time we access it the call to IoDeleteDevice was not already executed (so device object and its extension are freed)? – new_user Aug 03 '17 at 13:24
  • @new_user - if somebody send IO request to us - he must have pointer to DeviceObject and must be sure that he have reference to it (DeviceObject). so DeviceObject and DeviceExtension is valid (as memory storage). but state of DeviceObject - already another question - heve reference to object - not gurantee it state. only memory validity. and rundown protection used for protect part of object which can be destroyed **before** object itself will be destroyed. – RbMm Aug 03 '17 at 13:50
  • @new_user - for example i allocate memory buffer `devext->Buf` on start device. and io requests used this `Buf`. somebody have pointer with reference to my device (or even file opened on it) and send ioctl to it. the device object itself will be valid(as memory storage) until sender not release own reference(close file). even if we call `IoDeleteDevice` (memory will be freeded after last reference go to 0). but if we got remove request in concurent woth this ioctl and will not use rundown protection - we can free `Buf` until ioctl use it. – RbMm Aug 03 '17 at 14:14
  • the call `ExWaitForRundownProtectionRelease` give next - we wait if somebody use `Buf` at this time. and at once disable more use `Buf`. after this we can safe free `Buf` – RbMm Aug 03 '17 at 14:14
  • @RbMmGreat, thank you. Sorry if I get too curious, I still have one question. I have DispatchDeviceControl, DispatchCleanup, DispatchClose handlers. These requests are sent to bus FDO from user mode, in other words we have file opened on bus FDO (so validity of bus FDO and its extension as memory storage is guaranteed), but we need to access child PDO to complete the request (we use notion of device namespace): ChildPDO = (DEVICE_OBJECT*)((IoGetCurrentIrpStackLocation(Irp))->FileObject->FsContext); – new_user Aug 04 '17 at 13:37
  • @RbMmI fear that we will receive IRP_MN_REMOVE_DEVICE request while processing one of the aforementioned requests. Reference counter for child PDO will drop to zero and memory will be freed. I feel we need to create additional reference for child PDO and release it inside DispatchClose request handler, but I am not sure. – new_user Aug 04 '17 at 13:38
  • I think I understand. OS will increment reference counter on a given file object before it calls DispatchDeviceControl or DispatchCleaunup; OS will decrement reference counter on a given file object when we complete request. It means that DispatchDeviceControl and DispatchCleaunup can run concurrently, but DispatchClose cannot, we will get it only after we complete all another requests. Hence we are safe to release reference to child PDO in DispatchClose, while PnP manager will have its own reference. – new_user Aug 04 '17 at 14:19
  • in general you `FDO` maintain list of childs `PDO` and you remove `PDO` from list or when user close associated fileobject (you mark this pdo and call `IoInvalidateDeviceRelations`) or user select eject device. in this case first your pdo received `IRP_MN_REMOVE_DEVICE` (you not destroy it here because it enumerated, only mark. then fdo receive `BusRelations` - you remove this pdo from list and free reference on it. and then you fdo second time got `IRP_MN_REMOVE_DEVICE` - here you already delete it, because it not enumerated. i have exactly the same soft. – RbMm Aug 04 '17 at 14:29
  • @new_user you need protect access to pdo (via `FileObject->FsContext`) say by AcquirePushLockShared/ReleasePushLock on FDO. also AcquirePushLockShared/ReleasePushLock used when handle BusRelations. – RbMm Aug 04 '17 at 14:29
  • so on ioctl when you need access `PDO` (as `FileObject->FsContext` pointer) you enter some critsec - look are this stil not 0, and if yes - addref (if need send io to it) or mar as remove pending(on close or cleanup) and exit crit sec. then send request and release. or call `InvalidateDeviceRelations`. in handle `BusRelations` - you also enter the same critsec fro syncronize access to pdo list with user mode ioctls. this is separate from use rundown protection on PDO. so one protection on PDO (if need) another on `FDO`. this is most thin points. hard describe all on pure english here. :) – RbMm Aug 04 '17 at 14:34
  • @RbMmBy "select eject device" do you mean send "IOCTL_STORAGE_EJECT_MEDIA"? If so then am I correct that we do not need to remove PDO for devices such as disk drive (since it supports the notion of media ejection) and we need to remove PDO for devices such as USB stick (since it is itself "media")? Sorry if question is stupid. – new_user Aug 07 '17 at 16:22
  • @new_user - i mean user press `safely remove hardware` - in this case you not receiver any ioctl, but first your pdo received `IRP_MN_REMOVE_DEVICE` – RbMm Aug 07 '17 at 16:43
  • @RbMmThank you. And what should we do if bus device object receives `IRP_MN_SURPRISE_REMOVAL` and children are still connected? Will they automatically receive `IRP_MN_REMOVE_DEVICE` or we should mark all of them and call `IoInvalidateDeviceRelations`? – new_user Aug 12 '17 at 12:08
  • @new_user - you receive `IRP_MN_REMOVE_DEVICE` after `IRP_MN_SURPRISE_REMOVAL`. and you not need call `IoInvalidateDeviceRelations` in this case. you call it when(i assume you have virtual software device) user request inject or eject device. general rules for surprise removal https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/handling-an-irp-mn-surprise-removal-request removal - – RbMm Aug 12 '17 at 12:21
  • Thank you, I missed this one: "After this IRP succeeds and all open handles to the device are closed, the PnP manager sends an `IRP_MN_REMOVE_DEVICE` request to the device stack." So by the time we receive `IRP_MN_REMOVE_DEVICE` there will be no children attached, since we initiate child removal when we close handle or when we `safely remove hardware`. – new_user Aug 12 '17 at 13:29