-1

Im trying to resize an external process' window smaller than its minimum size constraint. What I tried to do was to inject a .dll and overwrite the WM_GETMINMAXINFO message, setting the ptMinTrackSize to {0,0}.

However this doesn't seem to work.

static LRESULT CALLBACK msghook(int code, WPARAM wParam, LPARAM lParam);

BOOL WINAPI DllMain(HINSTANCE hInst, DWORD reason, LPVOID reserved)
{
    switch (reason)
    {
    case DLL_PROCESS_ATTACH:
        hInstance = hInst;
        return TRUE;
    case DLL_PROCESS_DETACH:
        if (hwndServer != NULL)
        {
            clearMyHook(hwndServer);
        }
        return TRUE;
    }
}

_declspec(dllexport) BOOL setMyHook(HWND hwnd)
{
    if (hwndServer != NULL)
    {
        return FALSE;
    }
    hook = SetWindowsHookEx(WH_CALLWNDPROC, (HOOKPROC)msghook, hInstance, 0);
    if (hook != NULL)
    {
        hwndServer = hwnd;
        return TRUE;
    }
    return FALSE;
}

_declspec(dllexport) BOOL clearMyHook(HWND hwnd)
{
    if (hwnd != hwndServer)
    {
        return FALSE;
    }
    BOOL unhooked = UnhookWindowsHookEx(hook);
    if (unhooked)
    {
        hwndServer = NULL;
    }
    return unhooked;
}

static LRESULT CALLBACK msghook(int code, WPARAM wParam, LPARAM lParam)
{
    if (code < 0)
    {
        CallNextHookEx(hook, code, wParam, lParam);
        return 0;
    }

    LPMSG msg = (LPMSG)lParam;
    if (msg->message == WM_GETMINMAXINFO)
    {
        MINMAXINFO* mmInfo = (MINMAXINFO*)(lParam);
        mmInfo->ptMinTrackSize.x = 0;
        mmInfo->ptMinTrackSize.y = 0;
    }

    return CallNextHookEx(hook, code, wParam, lParam);
}

I'm then building my .exe including the .dll and linking the .lib and calling setMyHook() from there. I'm fairly new to the topic of .dll injection. Am I doing something wrong here, or is this simply not possible with the above approach?

Thanks..

proGrammar
  • 39
  • 1
  • 6

2 Answers2

0

The WH_CALLWNDPROC and WH_CALLWNDPROCRET hooks enable you to monitor messages sent to window procedures. The system calls a WH_CALLWNDPROC hook procedure before passing the message to the receiving window procedure, and calls the WH_CALLWNDPROCRET hook procedure after the window procedure has processed the message

Possibly the words "before" and "after" are the key to your problem :)

Jonathan Potter
  • 36,172
  • 4
  • 64
  • 79
0

Your code is based on assumptions that do not hold. This is spelled out in the documentation for CallWndProc:

The CallWndProc hook procedure can examine the message, but it cannot modify it.

The system stores the original message, and all changes to the message passed to the hook procedure are overwritten, before the message is handed to the window procedure. This applies to any memory referenced by the message, like the lParam pointing to a MINMAXINFO structure for a WM_GETMINMAXINFO message.

The same is true for CallWndRetProc:

The hook procedure can examine the message; it cannot modify it.

Since neither is useful in implementing your solution, your approach is destined to fail.

IInspectable
  • 46,945
  • 8
  • 85
  • 181
  • Damn, ok. So is there another solution to it? I thought about injecting the .dll and then subclassing the window by calling SetWindowLong and passing it a new WNDPROC method, would that work? Is there an easier solution? – proGrammar Dec 07 '15 at 13:08
  • @proGrammar: Subclassing is a potential solution. However, [SetWindowSubclass](https://msdn.microsoft.com/en-us/library/windows/desktop/bb762102.aspx) should be used instead of [SetWindowLongPtr](https://msdn.microsoft.com/en-us/library/windows/desktop/ms644898.aspx). You also need a `WH_CALLWNDPROC` or `WH_CALLWNDPROCRET` hook to get your code to run on the correct thread (i.e. the thread owning the window). – IInspectable Dec 07 '15 at 18:55
  • @IInspectable The hook can't modify the message itself, but if the message parameters point to an area of memory (as `lParam` does in `WM_GETMINMAXINFO`) there's nothing stopping the hook writing to that memory. – Jonathan Potter Dec 07 '15 at 21:08
  • @JonathanPotter: The pointer points to a copy. There's nothing stopping the hook from writing to that memory, but the changes are not reflected in the message passed to the window procedure. – IInspectable Dec 07 '15 at 21:13
  • @IInspectable So you're saying the hook subsystem makes a copy of any arbitrary data pointed to by the message? That seems a little far-fetched. It's the `MSG` structure that is copied, not whatever its parameters may or may not point to. – Jonathan Potter Dec 07 '15 at 21:14
  • @JonathanPotter: It's no more far-fetched than assuming, that the system will marshal system messages (including the memory pointed to) across process boundaries. It's a system message, the system knows how to handle those. – IInspectable Dec 07 '15 at 21:26
  • @JonathanPotter: I verified the behavior. Although the hook procedure and the window procedure do get the same pointer, any changes to the pointed to memory are lost once the hook procedure returns. Technically, the hook procedure doesn't get a copy, but rather the system stores a copy internally, and overwrites any changes to the message after the hook procedure returns. The effect is the same as if the hook procedure did get a copy. – IInspectable Dec 07 '15 at 22:08
  • @IInspectable Did you try with `WH_CALLWNDPROC` or `WH_CALLWNDPROCRET`? The point of my response to the OP (which perhaps was a little obtuse) was that `WH_CALLWNDPROC` is called *before* the target window, meaning the window proc itself overwrites any changes you made. It just doesn't make sense that the system would make **two** copies (backup + restore) of arbitrary message data for every message passed to the hook; I also believe the documentation doesn't claim that it does. – Jonathan Potter Dec 07 '15 at 22:57
  • @JonathanPotter: Changes made to the `MINMAXINFO` structure in the `WH_CALLWNDPROCRET` are discarded as well. When you send a `WM_GETMINMAXINFO` message to the window procedure, the values returned are those stored by the window procedure. The documentation does state, that you cannot modify the message for either hook. The term *message* apparently means *message including its data*. This applies to standard windows messages, and possibly user messages for known window classes (although I haven't verified this for user messages). – IInspectable Dec 07 '15 at 23:19