0

I want to hook certain API calls, e.g. CreateFile (or NtCreateFile if I hook ntdll.dll), but there are some issues.

I can use several methods to achieve this goal, e.g. DLL injection, Inline hooking etc. But I think the most pragmatic way would be to use Microsoft Detours, as it handles some functions, e.g. trampolines, for me. However, the issue is that I don't know which processes I want to hook beforehand, so my custom DLL should be loaded into the virtual address space of the process, when the path of the DLL is set to Kernel32.dll or when it specifically calls for the functions CreateFile, ReadFile etc.

However, since this is in the context of ransomware detection, I don't know which processes will invoke these API calls.. so actually I want to have a system-wide hook. I read articles etc. about this topic, and one of the approaches is to change the registry entries, but then it will only work for proceses that use user32.dll, have a GUI etc.

Can I anyone advise me on this case? Thank you in advance.

EDIT: I did some additional research. I think inline hooking makes the most sense. But unfortunately I can't use Detours for that, as this paper states (page 7), which I find a little bit strange because I would think that inline hooking is by definition system-wide (as each process that wants to use the function CreateFile from Kernel32.dll invokes the real DLL).

Moooz
  • 15
  • 5
  • Your issue has nothing to do with hooking, it's more so which process to hook and how to perform a global injection. In your case if you're trying to detect Ransomware , don't bother in user-mode. It's a waste of time. You should be writing a kernel device which hooks ZwCreateFile either directly or through the SSDT to detect any process in user-mode attempting to write. – Irelia Feb 02 '23 at 17:25
  • @Irelia developing a minifilter driver would indeed make more sense.. but I don't have experience with kernel programming at all etc. I actually want to avoid that (but maybe it's inevitable). But I would say (please correct me if I'm wrong) that if you can manage to hook all calls to certain Win32 API functions (e.g. CreateFile) within Kernel32.dll, it will still work, right? For sure, this approach is less solid than a kernel driver and a malicious process can evade this method easier, but... in principle it will work? – Moooz Feb 02 '23 at 17:30
  • It can work but malware and ransomware are sophisticated, they could easily detect hooks in their own process. Similarly they don't even have to use those functions, they can just set up the stack and registers themselves then pass the ZwCreateFile ordinal into a syscall which would effectively bypass your hook – Irelia Feb 02 '23 at 20:08
  • Thanks @Irelia, indeed they could bypass the system in several ways (the method you mention, also by using different API's, directly interact with the kernel etc.). In practice, however, most ransomware families use Win32, that doesn't of course mean that they cannot detect an API hook, so yeah that's quite :( – Moooz Feb 02 '23 at 20:16
  • I just wouldn't advice you writing a user mode global hook system. It's not effective and you'd also be hooking legitimate process. – Irelia Feb 02 '23 at 20:19

0 Answers0