MS Detours Express 3.0 is not hooking CreateFile win32 API function properly

Question

I am trying to hook win32 API function "CreateFile" using MS Detours, but when I test it by opening a *.doc file using MS Word, The CreateFile call for DLLs and font files and directories loaded by MS Word are redirected to my detoured function but not for that *.doc file, but when I open a *.txt file using notepad the CreateFile call for that *.txt file comes to my detoured function.

I am using following code for hooking CreateFile:

static HANDLE (WINAPI *Real_CreateFile)(LPCWSTR lpFileName, DWORD dwDesiredAccess, DWORD dwShareMode, LPSECURITY_ATTRIBUTES lpSecurityAttributes, DWORD dwCreationDisposition, DWORD dwFlagsAndAttributes, HANDLE hTemplateFile) = CreateFile;

HANDLE WINAPI Routed_CreateFile(LPCWSTR lpFileName, DWORD dwDesiredAccess, DWORD dwShareMode, LPSECURITY_ATTRIBUTES lpSecurityAttributes, DWORD dwCreationDisposition, DWORD dwFlagsAndAttributes, HANDLE hTemplateFile)
{
OutputDebugString(lpFileName);
return Real_CreateFile(lpFileName, dwDesiredAccess, dwShareMode, lpSecurityAttributes, dwCreationDisposition, dwFlagsAndAttributes, hTemplateFile);
}

BOOL APIENTRY DllMain( HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved )
{
LONG Error;
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:

    OutputDebugString(L"Attaching MyDLL.dll");
    OutputDebugString(strInfo);
    DetourRestoreAfterWith();
    DetourTransactionBegin();
    DetourUpdateThread(GetCurrentThread());
    DetourAttach(&(PVOID&)Real_CreateFile, Routed_CreateFile);
    Error = DetourTransactionCommit();

    if (Error == NO_ERROR)
        OutputDebugString(L"Hooked Success");
    else
        OutputDebugString(L"Hook Error");

    break;
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
    OutputDebugString(L"De-Attaching MyDLL.dll");
    DetourTransactionBegin();
    DetourUpdateThread(GetCurrentThread());
    DetourDetach(&(PVOID&)Real_CreateFile, Routed_CreateFile);
    Error = DetourTransactionCommit();

    if (Error == NO_ERROR)
        OutputDebugString(L"Un-Hooked Success");
    else
        OutputDebugString(L"Un-Hook Error");

    break;
}
return TRUE;
}

Thanks in advance.

I'm guessing _MS Word_ doesn't use `CreateFile` then to load _.doc_ s — K-ballo, Jan 09 '13 at 12:50
@ K-ballo: As seen in Process Monitor, MS Word makes a call to CreateFile for loading *.doc file. — Ahsan Raza, Jan 09 '13 at 13:13
Office programs are very attractive malware targets due to their wide-spread use. Visible from the constant security updates. A program like Word is going to have counter-measures against them. Pretty unprovable of course, beyond seeing detouring not working as you'd expect. — Hans Passant, Jan 09 '13 at 13:17
Have you tested your code with an application (e.g. one you created yourself) that is known the _really_ use `CreateFile`? — Christian.K, Jan 09 '13 at 13:51

score 3 · Accepted Answer · answered Jan 09 '13 at 13:47

I think you are missing a break after this:

case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
    break;  // Not interested in thread messages
case DLL_PROCESS_DETACH:

Are you just detaching the detour before it is called? Maybe opening a .doc creates a new thread but a .txt doesn't, triggering this code path.

score 1 · Answer 2 · answered Jan 10 '13 at 16:06

1

It looks like you're not initializing your Real_CreateFile function pointer properly. I'm guessing you're setting it to your module's import table entry for CreateFile.

Instead, initialize it to GetProcAddress(GetModuleHandle("kernel32"),"CreateFileW");

answered Jan 10 '13 at 16:06

Marc Sherman

2,303
14
22

MS Detours Express 3.0 is not hooking CreateFile win32 API function properly

2 Answers2

Linked