1

Question Purpose: Reality check on the MS docs of DllMain.

It is "common" knowledge that you shouldn't do too much in DllMain, there are definite things you must never do, some best practises.

I now stumbled over a new gem in the docs, that makes little sense to me: (emph. mine)

When handling DLL_PROCESS_DETACH, a DLL should free resources such as heap memory only if the DLL is being unloaded dynamically (the lpReserved parameter is NULL). If the process is terminating (the lpvReserved parameter is non-NULL), all threads in the process except the current thread either have exited already or have been explicitly terminated by a call to the ExitProcess function, which might leave some process resources such as heaps in an inconsistent state. In this case, it is not safe for the DLL to clean up the resources. Instead, the DLL should allow the operating system to reclaim the memory.

Since global C++ objects are cleaned up during DllMain/DETACH, this would imply that global C++ objects must not free any dynamic memory, because the heap may be in an inconsistent state. / When the DLL is "linked statically" to the executable. / Certainly not what I see out there - global C++ objects (iff there are) of various (ours, and third party) libraries allocate and deallocate just fine in their destructors. (Barring other ordering bugs, o.c.)

So, what specific technical problem is this warning aimed at?

Since the paragraph mentions thread termination, could there be a heap corruption problem when some threads are not cleaned up correctly?

Martin Ba
  • 37,187
  • 33
  • 183
  • 337
  • This kind of verbiage gets added when they had one too many support phone calls. The only reason to document an undocumented parameter. An example scenario is a programmer using SetUnhandledExceptionFilter(), do something useful to help diagnose the crash, and then ExitProcess instead of TerminateProcess. Always works when tested, bummer when in the real world the crash was caused by heap corruption and the heap lock is still held. – Hans Passant Dec 17 '18 at 17:06

2 Answers2

1

The ExitProcess API in general does the follwoing:

  • Enter Loader Lock critical section
  • lock main process heap (returned by GetProcessHeap()) via HeapLock(GetProcessHeap()) (ok, of course via RtlLockHeap) (this is very important step for avoid deadlock)
  • then terminate all threads in process, except current (by call NtTerminateProcess(0, 0) )
  • then call LdrShutdownProcess - inside this api loader walk by loaded module list and sends DLL_PROCESS_DETACH with lpvReserved nonnull.
  • finally call NtTerminateProcess(NtCurrentProcess(), ExitCode ) which terminates the process.

The problem here is that threads terminated in arbitrary place. For example, thread can allocate or free memory from any heap and be inside heap critical section, when it terminated. As a result, if code during DLL_PROCESS_DETACH tries to free a block from the same heap, it deadlocks when trying to enter this heap's critical section (if of course heap implementation use it).

Note that this does not affect the main process heap, because we call HeapLock for it before terminate all threads (except current). The purpose of this: We wait in this call until all another threads exit from process heap critical section and after we acquire the critical section, no other threads can enter it - because the main process heap is locked.

So, when we terminate threads after locking the main heap - we can be sure that no other threads that are killed are inside main heap critical section or heap structure in inconsistent state. Thanks to RtlLockHeap call. But this is related only to main process heap. Any other heaps in the process are not locked. So these can be in inconsistent state during DLL_PROCESS_DETACH or can be exclusively acquired by an already terminated thread.

So - using HeapFree for GetProcessHeap or saying LocalFree is safe (however not documented) here.

Using HeapFree for any other heaps is not safe if DllMain is called during process termination.

Also if you use another custom data structures by several threads - it can be in inconsistent state, because another threads (which can use it) terminated in arbitrary point.

So this note is warning that when lpvReserved parameter is non-NULL (what is mean DllMain is called during process termination) you need to be especially careful in clean up the resources. Anyway all internal memory allocations will be free by operation system when process died.

Martin Ba
  • 37,187
  • 33
  • 183
  • 337
RbMm
  • 31,280
  • 3
  • 35
  • 56
  • Awesome! What you write makes a lot of sense. May I inquire how you know how the `ExitProcess` function operates? Is this documented somewhere? – Martin Ba Dec 17 '18 at 20:15
  • @MartinBa - this is very easy look under debugger. but sure no any formal documentation from ms – RbMm Dec 17 '18 at 21:09
0

As an addendum to RbMm's excellent answer, I'll add a quote from ExitProcess that does a much better job - than the DllMain docs do - at explaining, why heap operation (or any operation, really) can be compromised:

If one of the terminated threads in the process holds a lock and the DLL detach code in one of the loaded DLLs attempts to acquire the same lock, then calling ExitProcess results in a deadlock. In contrast, if a process terminates by calling TerminateProcess, the DLLs that the process is attached to are not notified of the process termination. Therefore, if you do not know the state of all threads in your process, it is better to call TerminateProcess than ExitProcess. Note that returning from the main function of an application results in a call to ExitProcess.

So, it all boils down to: IFF you application has "runaway" threads that may hold any lock, the (CRT) heap lock being a prominent example, you have a big problem during shutdown, when you need to access the same structures (e.g. the heap), that your "runaway" threads are using.

Which just goes to show that you should shut down all your threads in a controlled way.

Martin Ba
  • 37,187
  • 33
  • 183
  • 337