Incorrect listview hittesting

Question

INTRODUCTION:

I am trying to determine if user clicked on item or above/below listview.

Listview is in report mode, with extended styles full row select and grid lines.

PROBLEM:

I am not able to get correct results, based on the docs for LVHITTESTINFO.

MY EFFORTS TO SOLVE THE PROBLEM:

I have made checkboxes with same caption as the values in LVHITTESTINFO in my main window.

These are LVHT_ABOVE, LVHT_BELOW, LVHT_NOWHERE, LVHT_TOLEFT, LVHT_TORIGHT and LVHT_ONITEM.

My goal is to check the ones with same caption as returned result from hittesting.

I have captured the mouse in my main window in response to WM_LBUTTONDOWN and am doing the hittesting in WM_LBUTTONUP handler.

This was the easiest way for me to code the smallest SSCCE/code snippets that can be posted here.

Here is the relevant code:

case WM_LBUTTONDOWN:
    // reset checkboxes
    CheckDlgButton(hWnd, 3000, BST_UNCHECKED);
    CheckDlgButton(hWnd, 3100, BST_UNCHECKED);
    CheckDlgButton(hWnd, 3200, BST_UNCHECKED);
    CheckDlgButton(hWnd, 3300, BST_UNCHECKED);
    CheckDlgButton(hWnd, 3400, BST_UNCHECKED);
    CheckDlgButton(hWnd, 3500, BST_UNCHECKED);
    // capture the mouse
    SetCapture(hWnd);
    break;
case WM_LBUTTONUP:
{
    // extract coordinates
    POINT pt = { 0 };
    pt.x = GET_X_LPARAM(lParam);
    pt.y = GET_Y_LPARAM(lParam);
    // do the hittesting
    ClientToScreen(hWnd, &pt);
    ScreenToClient(GetDlgItem(hWnd, 2000), &pt);

    LVHITTESTINFO lvhti = { 0 };
    lvhti.pt = pt;

    ListView_HitTest(GetDlgItem(hWnd, 2000), &lvhti);
    // check appropriate checkboxes
    if ((lvhti.flags & LVHT_ABOVE) == LVHT_ABOVE)
        CheckDlgButton(hWnd, 3000, BST_CHECKED);
    if ((lvhti.flags & LVHT_BELOW) == LVHT_BELOW)
        CheckDlgButton(hWnd, 3100, BST_CHECKED);
    if ((lvhti.flags & LVHT_NOWHERE) == LVHT_NOWHERE)
        CheckDlgButton(hWnd, 3200, BST_CHECKED);
    if ((lvhti.flags & LVHT_ONITEM) == LVHT_ONITEM)
        CheckDlgButton(hWnd, 3300, BST_CHECKED);
    if ((lvhti.flags & LVHT_TOLEFT) == LVHT_TOLEFT)
        CheckDlgButton(hWnd, 3400, BST_CHECKED);
    if ((lvhti.flags & LVHT_TORIGHT) == LVHT_TORIGHT)
        CheckDlgButton(hWnd, 3500, BST_CHECKED);
    // release mouse capture
    ReleaseCapture();
}
    break;

TESTING PRINCIPLE:

Testing is done in the following way: I click on main windows client area, hold left mouse button down, and drag cursor over item/above (or below) listview. Then I release button which invokes WM_LBUTTONUP message and appropriate checkboxes get checked.

TESTING RESULTS:

Tested on Windows 7

• When releasing left mouse button over item's first subitem nothing is checked.
• When releasing left mouse button over item's second ( third, etc ) subitem checkboxes LVHT_ABOVE and LHVT_ONITEM are checked.
• When releasing button outside of listview I correctly get LVHT_NOWHERE.
• When releasing button over scrollbars I get LVHT_NOWHERE.
• When releasing above header control's first column I get nothing checked, yet for other columns I get LVHT_ONITEM and LVHT_ABOVE.

Tested on Windows XP

• When releasing left mouse button over item's first subitem nothing is checked.
• When releasing left mouse button over item's second ( third, etc ) subitem checkboxes LVHT_ABOVE and LHVT_ONITEM are checked.
• When releasing button outside of listview I get correct combination of LVHT_ABOVE/LVHT_BELOW and LVHT_TOLEFT/LVHT_TORIGHT.
• When releasing button over scrollbars I get LVHT_TORIGHT for vertical, and LVHT_BELOW for horizontal scrollbar.
• When releasing above header control I get inconsistent behavior: When releasing at the very top of the header's item I get LVHT_NOWHERE, yet in other cases it behaves as same as I have described above ( for Windows 7 ).

Important:

• Searching through Internet, I have found out that LVHT_ABOVE and LVHT_ONITEMSTATEICON have the same value. This means that I must interpret testing results where I got LVHT_ABOVE and LVHT_ONITEM as LVHT_ONITEM.
• While debugging, I have found out that when releasing mouse button above first column or first item gives mask 4, meaning I get LVHT_ONITEMLABEL instead of LVHTI_ONITEM.

QUESTION:

How can I adjust the above code to get correct hittesting results?

If the solution to my problem is complex and requires too much space and effort for you to post, I will be satisfied with instructions and guidelines or atleast some links that point me in the right direction.

If further info is required I will update my post.

Answer 1

Because LVHT_ONITEMSTATEICON and LVHT_ABOVE have the same value, you have to look at the requested y coordinate to differentiate between them. LVHT_ABOVE means the hit result is above the client area, so it can only be set when the y coordinate is less than zero, whereas LVHT_ONITEMSTATEICON can only be set when the hit result is on an icon within the client area.

The LVHT_ONITEM... flags are only meaningful when LVHT_ONITEM is not present. LVHT_ONITEM means the hit result is on the overall item as a whole, whereas LVHT_ONITEM... mean the hit result is on a particular piece of an item.

Try this:

if (((lvhti.flags & LVHT_ABOVE) == LVHT_ABOVE) && (pt.y < 0))
    // above the client area ...

...

if ((lvhti.flags & LVHT_ONITEM) == LVHT_ONITEM)
    // on an item as a whole...
else
{
    if ((lvhti.flags & LVHT_ONITEMICON) == LVHT_ONITEMICON)
        // on an item's icon...
    if ((lvhti.flags & LVHT_ONITEMLABEL) == LVHT_ONITEMLABEL)
        // on an item's text...
    if ((lvhti.flags & LVHT_ONITEMSTATEICON) == LVHT_ONITEMSTATEICON)
        // on an item's state icon...
}

Also, don't forget to look at the return value of ListView_HitTest(), and/or the LVHITTESTINFO::iItem field. They tell you the list item index, or -1 if the hit result is not on an item.