64-bit global hook locks up 32-bit applications

Question

I am attempting to create a simple DLL to hook into 64-bit applications on Win7 x64.

My program works with 64-bit processes, spawning a message box whenever a key is pressed. But when I press any key in a 32-bit application, that application will lock up until the hook is removed. Why does the 64-bit hook interfere with 32-bit applications?

Code below.

hook.h

#ifdef MYHOOK_EXPORTS
#define MYHOOK_DLL __declspec(dllexport) __stdcall
#else
#define MYHOOK_DLL __declspec(dllimport) __stdcall
#endif

void MYHOOK_DLL installHook();
void MYHOOK_DLL removeHook();

hook.cpp:

#include "hook.h"
#include <windows.h>

// hook handle stored in a shared data segment
#pragma data_seg(".myshared")
HHOOK hhook = 0;
#pragma data_seg()
#pragma comment(linker, "/SECTION:.myshared,RWS")

static HMODULE hmodule = 0;

LRESULT CALLBACK keyboardProc(int code, WPARAM wParam, LPARAM lParam) {
    // Replace this with sending a message to another window or writing to a file
    MessageBox(NULL, L"Hello, world!", L"Alert", 0);

    return CallNextHookEx(hhook, code, wParam, lParam);
}

void MYHOOK_DLL installHook() {
    if (!hhook)
        hhook = SetWindowsHookEx(WH_KEYBOARD, keyboardProc, hmodule, 0);
}

void MYHOOK_DLL removeHook() {
    if (hhook)
        UnhookWindowsHookEx(hhook);
    hhook = 0;
}

BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call,
    LPVOID lpReserved) {

    switch (ul_reason_for_call) {
    case DLL_PROCESS_ATTACH:
        hmodule = hModule;
    }
    return TRUE;
}

install.cpp:

#include "hook.h"
#include <stdio.h>

int wmain() {
    installHook();
    getchar();
    removeHook();

    return 0;
}

I've read you should check if you're in a 64-bit process somewhere in the hook, but I'm not sure how and where.

I'm aware of WH_KEYBOARD_LL which doesn't require injecting a DLL, but I'm trying to understand why this hook isn't working.

score 1 · Answer 1 · answered Jun 23 '11 at 03:08

1

a) you can't call MessageBox from a Hook procedure. Don't do that. b) Hooks are almost always the wrong way to solve any problem. I recommend against them. What are you really trying to do?

answered Jun 23 '11 at 03:08

Martyn Lovell

2,086
11
13

I'm trying to implement the canonical example of a WH_KEYBOARD hook on a 64-bit system, this isn't part of anything bigger. – acdx Jun 23 '11 at 03:31
Try just capturing the keys you see into a simple array. That should be safe. – Martyn Lovell Jun 24 '11 at 00:40
@Martyn: Tried that. It records keystrokes from 64-bit apps fine, and 32-bit apps lock up. The hook procedure never gets executed, because obviously it's 64-bit and the program is 32-bit. – acdx Jun 24 '11 at 04:33
You can't have one hook DLL both for 32 and 64 bit processes. Give them two different names, and hook once for each in the right kind of process. See http://msdn.microsoft.com/en-us/library/ms644990(VS.85).aspx – Martyn Lovell Jun 24 '11 at 04:48
@Martyn, that's precisely the problem. How do I ask it to globally hook only 64-bit processes? If I compile and run two versions of the program, with different DLL names, then ALL programs will hang on keyboard input. – acdx Jun 24 '11 at 05:15
When you call from a 64 bit process, you only impact other 64 bit processes. And if everything hung, that's probably because your hook made some core 64 bit process (e.g. explorer) go bad. – Martyn Lovell Jun 24 '11 at 06:58
1

I've found four people with the same problem ([1](http://stackoverflow.com/questions/5627559/32-bit-keyboard-hook-unexpectedly-works-in-64-bit-apps-but-is-hanging-outlook), [2](http://social.msdn.microsoft.com/Forums/pl-PL/windowsgeneraldevelopmentissues/thread/7d2353e2-5108-479b-b886-4c437912c9e5), [3](http://social.msdn.microsoft.com/Forums/en/windowsgeneraldevelopmentissues/thread/236a1e72-0ec3-471d-88a7-572462a5d684), [4](http://social.msdn.microsoft.com/Forums/en/windowssdk/thread/0e69d013-aff2-42f1-96ae-44778b9792d1)). If you do know a solution, I'll definitely checkmark it. – acdx Jun 24 '11 at 15:19

score 0 · Answer 2 · answered Jul 21 '15 at 09:42

Modify your install.cpp as follows.

#include "hook.h"
#include <stdio.h>

int wmain() {
    installHook();
    MSG msg;
    BOOLEAN bRunThread = TRUE;
    while(bRunThread)
    {
        PeekMessage(&msg, NULL, 0, 0, PM_REMOVE);
        TranslateMessage(&msg);
        DispatchMessage(&msg);
    }

    removeHook();

    return 0;
}

64-bit global hook locks up 32-bit applications

2 Answers2