SetWindowsHookEx succeed but callback function is never called

Question

dll code:

LRESULT CALLBACK CBTNewProc(int nCode, WPARAM wParam, LPARAM lParam)
{

    std::ofstream file;
    file.open("E:\\enter.txt", std::ios::out);
    file << nCode;
    file.close();

    return CallNextHookEx(g_hHook, nCode, wParam, lParam);
}


extern "C" __declspec(dllexport) void installHook()
{
    if (g_hHook != NULL){
        UnhookWindowsHookEx(g_hHook);
        g_hHook = NULL;
    }
    HINSTANCE hInstance = GetModuleHandle(NULL);
    g_hHook = SetWindowsHookEx(WH_CBT, CBTNewProc, NULL, GetCurrentThreadId());
    if (g_hHook == NULL)
    {
        MessageBox(NULL, L"fail!", L"caption", MB_OK);
    }
    else
    {
        MessageBox(NULL, L"install success!", L"caption", MB_OK);
    }
}

I wrote another program to load this dll and called installHook. The message box "install success" is showed but the callback function was never called, enter.txt is not found under drive E.

I'm using Win7 + VS2013.

It might be worth double-checking that the callback function really isn't being called, e.g., by changing it so that it crashes the process. (Just in case it's actually a problem writing to the file, or the C++ runtime is malfunctioning, or anything like that.) — Harry Johnston, Oct 10 '15 at 07:37
@HarryJohnston Hi, I tried showing a message box inside callback function and no message box showed up at runtime — Wood, Oct 10 '15 at 08:09
You are setting a single hook on a single thread. Is it the right thread? — Ben, Oct 10 '15 at 09:55
@Ben You are right, I change the params of SetWindowsHookEx to WH_CBT, CBTNewProc, hInstance, GetCurrentThreadId() and problem is solved. Thx~ — Wood, Oct 10 '15 at 10:01
Note that `GetModuleHandle(NULL)` returns the module handle of the calling process, not the DLL. You need to use the DLL's module handle instead. Your DLL gets that from its `DllMain()`/`DllEntryPoint()` function. — Remy Lebeau, Oct 10 '15 at 20:03
@Wood: that's odd; according to the documentation, if you specify a thread in the current process you *shouldn't* specify `hInstance`. — Harry Johnston, Oct 10 '15 at 23:56
@HarryJohnston I want to hook system event, not just current process :-) — Wood, Oct 12 '15 at 02:32
Then the thread ID should be zero, as per Ben's answer. (Though I'm still puzzled as to why the hook doesn't kick in when you call MessageBox.) — Harry Johnston, Oct 12 '15 at 20:49

score 0 · Accepted Answer · answered Oct 10 '15 at 14:25

0

For a hook to be set in other processes, you must pass the hInstance of the DLL which contains the hook proc to SetWindowsHookEx.

You should also pass zero as the thread ID.

answered Oct 10 '15 at 14:25

Ben

34,935
6
74
113

SetWindowsHookEx succeed but callback function is never called

1 Answers1