0

I've been looking at some hooking code which selectively loads a library into certain processes and then hooks certain native API functions (using Detours). The chain of events looks like this:

  • Kernel driver loads A.dll into every process.
  • A.dll::DllMain() decides whether to load B.dll (LoadLibraryEx) which contains actual Detours hooks.
  • B.dll runs for the duration of the process hooking said functions.

The second bullet here appears to break the DllMain rules specified here, but I'm trying to work out if the way the driver loads A.dll works around the limitations. Specifically, the kernel driver uses PsSetLoadImageNotifyRoutine to get notifications when each process starts and then queues an APC to call LoadLibraryEx on A.dll which means it's pretty much the first DLL loaded when the process starts. Does this circumvent the problems with calling LoadLibrary within DllMain?

Benj
  • 31,668
  • 17
  • 78
  • 127

2 Answers2

2

Doesn't matter how the LoadLibraryEx was triggered. Once triggered, the DLL loading process is the same, and the same rules apply.

Raymond Chen
  • 44,448
  • 11
  • 96
  • 135
  • It seems to me that even the first step might be hazardous; is there a risk of the APC loading A.dll getting executed while another DLL is in the process of being loaded? – Harry Johnston Oct 30 '12 at 20:48
  • Yes, this is all very dodgy and in no way supported. – Raymond Chen Oct 30 '12 at 21:33
  • @RaymondChen - So what would be your recommended approach for inserting a detours hooking dll into a process and deciding which processes to hook based on user land config info? (Or is this where you tell me that you don't approve of detours at all) :-) – Benj Oct 30 '12 at 22:30
  • Detouring processes without their permission is totally unsupported. This is the sort of thing that gets flagged by anti-virus software as malware. – Raymond Chen Oct 30 '12 at 23:03
  • @RaymondChen I'm not sure about that last bit, virus scanners definitely look at CreateRemoteThread etc as part of their heuristics engines. But MS license Detours, and it's used by a bunch of large MS business parters such as Citrix and Symantec. If it was automatically flagged by virus scanners, the guys who work in various areas of virtualization would be in big trouble. In actual fact, virus scanner manufacturers work with such vendors in order to distinguish between various types of hook injection behaviour. – Benj Oct 31 '12 at 07:54
  • I've also discovered that the hooking code I'm looking at is well aware that it's breaking the rules. Both A.dll and B.dll are written carefully to use only the native API and not to implicitly load other DLLs so as not to create a loader loop. – Benj Oct 31 '12 at 14:25
  • @Benj I'm not talking about Detours. I'm talking about pre-process code injection. Valid Detours usage is where an application detours itself, not when an external agent injects into a process in order to detour it. That smells virusy. – Raymond Chen Oct 31 '12 at 14:45
  • @RaymondChen Many of the Detours samples involve detouring other executables without their knowledge. For example, in order to trace memory allocations or registry accesses of third party programs. I know of at least 2 major virtualization technologies that make use of it (not including the one I work with). – Benj Oct 31 '12 at 15:00
  • That's the difference between a sample and production code. Samples can do things that cannot be shipped in production. – Raymond Chen Oct 31 '12 at 15:42
  • Haha, define "cannot" ;-) Hooking is big business even if it does offend your sensibilities. – Benj Oct 31 '12 at 15:50
  • "Cannot" in the sense of "will not be supported if you do." Make sure your customers understand this. – Raymond Chen Oct 31 '12 at 17:38
  • My experience of working with MS over the years is that they're not unreasonable about issues like this. If a problem is thought to be as a result of our use of undocumented APIs etc, then we're expected to provide the support. If it's shown to be an issue that's unrelated to that, then MS are usually willing to provide support. Usually it's the size the sales opportunity etc which truly governs the willingness on both sides. We often make a call whether the risk associated with undocumented stuff outweighs the commercial value of offering features that can't be provided another way. – Benj Oct 31 '12 at 22:55
1

The documentation very specifically says not to call LoadLibrary in DllMain. Even in the unlikely event that you figured out a safe way to make it work, it may not work in the next version (or even the next service pack) of Windows.

Mark Wilkins
  • 40,729
  • 5
  • 57
  • 110