How to extract colors from NM_CUSTOMDRAW message in Win32?

Question

I have a hooked dll with WH_CALLWNDPROC from a x86 program (MSVS 2017). The code goes as follows:

extern "C" __declspec(dllexport) LRESULT SysMessageProc(int nCode, WPARAM wParam, LPARAM lParam)
{
if (nCode < 0) return CallNextHookEx(NULL, nCode, wParam, lParam);

    if (nCode == HC_ACTION) {

        CWPSTRUCT* pCWP = (CWPSTRUCT*)lParam;

        HWND hWnd = pCWP->hwnd;

        char wclass[256]; wclass[0] = 0;
        if (GetClassNameA(hWnd, wclass, 255) != 0) {
            
            if (pCWP->message == WM_NOTIFY && (strcmp(wclass, "SysHeader32") == 0 || strcmp(wclass, "SysListView32") == 0)) {

                // LPNMLISTVIEW pnm = (LPNMLISTVIEW)pCWP->lParam;
                // pnm->lParam is always 0

                NMHDR* hdr = (NMHDR*)pCWP->lParam;

                if ((int)hdr->code == NM_CUSTOMDRAW) {

                    char out[256]; out[0] = 0;
                    sprintf_s(out, 255, 
                        "class: '%s', hWnd: %08X, msg: %08X, ptr: %08X, HDR[%08X, %i, %i]\n",
                        wclass, (UINT)hWnd, pCWP->message, (UINT)pCWP->lParam,
                        (UINT)hdr->hwndFrom, hdr->idFrom, (int)hdr->code);

                    FILE *stream;
                    if (fopen_s(&stream, "F:\\tmp\\_dll\\out.txt", "a+") == 0) {
                        fprintf(stream, out);
                        fclose(stream);
                    }

                   // how to get
                   // LPNMLVCUSTOMDRAW lplvcd = ???;
                   // lplvcd->clrFace, lplvcd->clrText, lplvcd->clrTextBk ???
                   

               }

          }

     }

When I record messages I get the same results as in Spy++:

It seems strange that hdr->code is UINT and NM_CUSTOMDRAW = -12 so I have to convert to (int) but the result matches the output from Spy++.
Initially I thought that I can do this:

LPNMLISTVIEW pnm = (LPNMLISTVIEW)pCWP->lParam;
...
LPNMLVCUSTOMDRAW lplvcd = (LPNMLVCUSTOMDRAW)pnm->lParam;
...
print(lplvcd->clrFace, lplvcd->clrText, lplvcd->clrTextBk);

but pnm->lParam is always zero; pCWP->wParam is also zero.

How to I get to LPNMLISTVIEW and LPNMLVCUSTOMDRAW structures ?

EDIT:

CWPSTRUCT* pCWP = (CWPSTRUCT*)lParam;
HWND hWnd = pCWP->hwnd;
UINT msg = pCWP->message;

char wclass[256]; wclass[0] = 0;
if (RealGetWindowClassA(hWnd, wclass, 255) != 0) {
                
    if (msg == WM_NOTIFY && (strcmp(wclass, "SysHeader32") == 0 || strcmp(wclass, "SysListView32") == 0)) {

        NMHDR* hdr = (NMHDR*)pCWP->lParam;

        if (hdr->code == NM_CUSTOMDRAW) {
            
            if (strcmp(wclass, "SysListView32") == 0) {

                LPNMLVCUSTOMDRAW lplvcd = (LPNMLVCUSTOMDRAW)pCWP->lParam;
                
                if (lplvcd) {
                    RECT r = lplvcd->rcText;
                    char out[512]; out[0] = 0;
                    sprintf_s(out, 511, 
                    "class: '%s', hWnd: %08X, msg: %08X, ptr: %08X, \
draw: %08X, rect: (%i, %i)-(%i, %i), face: %08X, txt: %08X, txtBg: %08X\n",
                        wclass, (UINT)hWnd, msg, (UINT)lplvcd, 
                        lplvcd->nmcd.dwDrawStage, r.left, r.top, r.right, r.bottom, 
                        lplvcd->clrFace, lplvcd->clrText, lplvcd->clrTextBk);
                    
                    FILE *stream;
                    if (fopen_s(&stream, "F:\\tmp\\_dll\\out.txt", "a+") == 0) {
                        fprintf(stream, out);
                        fclose(stream);
                    }
                }
            }

Example output:

class: 'SysListView32', hWnd: 000A04F4, msg: 0000004E, ptr: 02E3F33C, draw: 00000001, rect: (0, 1)-(-1241442379, 48493508), face: FFB617B5, txt: 02E3F400, txtBg: 6F010726

FYI, `GetClassName()` may not return what you want for subclassed controls. Consider using [`RealGetWindowClass()`](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-realgetwindowclassa) instead. See [What makes RealGetWindowClass so much more real than GetClassName?](https://devblogs.microsoft.com/oldnewthing/20101231-00/?p=11863). — Remy Lebeau, Jan 08 '21 at 17:45

Remy Lebeau · Answer 1 · 2021-01-08T18:02:57.757

1

You are accessing NMLVCUSTOMDRAW from the wrong lParam.

In a ListView's NM_CUSTOMDRAW notification, the wParam is always 0, and the lParam is a NMLVCUSTOMDRAW*, but you are casting that to NMLISTVIEW* instead, which is wrong. There is no NMLISTVIEW provided in NM_CUSTOMDRAW. And even if there were, the NMLISTVIEW::lParam field contains the ListView item's user-defined data, as set by the application via LVM_SETITEM/ListView_SetItem(). So that is not what you want anyway.

You need to cast pCWP->lParam directly to NMLVCUSTOMDRAW* instead. And only for a SysListView32 control. A SysHeader32 control uses NMCUSTOMDRAW* instead, eg:

if (pCWP->message == WM_NOTIFY) {
    ...
    NMHDR* hdr = (NMHDR*)pCWP->lParam;
    ...
    if (hdr->code == NM_CUSTOMDRAW) {
        if (strcmp(wclass, "SysListView32") == 0) {
            LPNMLVCUSTOMDRAW lplvcd = (LPNMLVCUSTOMDRAW)pCWP->lParam;
            // use lplvcd as needed...
        }
        else if (strcmp(wclass, "SysHeader32") == 0) {
            LPNMCUSTOMDRAW lpcd = (LPNMCUSTOMDRAW)pCWP->lParam;
            // use lpcd as needed...
        }
        ...
    }
}

The documentation for the general-purpose NM_CUSTOMDRAW shows which struct is used by which type of custom-drawable control. There are 5 possibilities:

edited Jan 08 '21 at 18:02

answered Jan 08 '21 at 17:52

Remy Lebeau

555,201
31
458
770

Thank you, I made changes, please see edit. Is it correct now? Because the list view is a standard looking black text on white background element and the color values that I get do not make sense as well as the rectangle coordinates. The color values are different in every message except clrText and drawstage is always 1. – Jan 09 '21 at 14:01
DrawStage 1 is `CDDS_PREPAINT`, I don't know if the fields you are accessing are even valid in that stage or not. I don't use those fields in that stage, and no example I have found online does, either. – Remy Lebeau Jan 11 '21 at 17:51
What might be the reason I don't get messages with other draw stages? I tried it on two different programs with list views. – Jan 11 '21 at 21:56
That entirely depends on what value the receiving application is returning in reply to `NM_CUSTOMDRAW`. [Read the documentation](https://learn.microsoft.com/en-us/windows/win32/controls/nm-customdraw-list-view). What is the return value being set to? I'm assuming `CDRF_DODEFAULT` (0), which would disable all draw notifications other than `CDDS_PREPAINT`. You won't be able to get that return value from a `WH_CALLWNDPROC` hook, you will have to use a `WH_CALLWNDPROCRET` hook instead. – Remy Lebeau Jan 11 '21 at 23:00
OK, thanks, I'll try to dig further into it – Jan 12 '21 at 08:02

How to extract colors from NM_CUSTOMDRAW message in Win32?

1 Answers1