How to cancel item label editing in Tree-View control upon ESC keydown in WinAPI

Question

I have a dialog box with a Tree-View control where the user can edit the item labels. I want the user to be able to cancel the label edit by pressing ESC key.

The problem is that pressing ESC closes the dialog window immediately.

I have tried getting the handle to the EditBox control by a TreeView_GetEditControl() call upon TVN_BEGINLABELEDIT message and subclassing it to trap the ESC key, but when I do that, typing in edit box becomes impossible.

What is the problem?

Here is the relevant code:

INT_PTR CALLBACK DlgProc(HWND hWnd, UINT message, 
                         WPARAM wParam, LPARAM lParam) {
    switch(message) {
        //...

        case WM_NOTIFY:
        {
            LPNMHDR pNmHdr = (LPNMHDR)lParam;
            switch(pNmHdr->code) {
                case TVN_BEGINLABELEDIT:
                {
                    HWND hwndTV = (HWND)GetWindowLongPtr(hWnd, GWLP_USERDATA); // stored handle to Tree-View ctl
                    HWND hWndEditBox = TreeView_GetEditControl(hwndTV);

                    // subclass edit box
                    TreeViewGlobals::g_wpOrigEditBoxProc =
                        (WNDPROC)SetWindowLongPtr(hWndEditBox, 
                                                  GWLP_WNDPROC, (LONG_PTR)EditBoxCtl_SubclassProc);
                    break;
                }
                case TVN_ENDLABELEDIT:
                {
                    SetWindowLongPtr(hWnd, DWLP_MSGRESULT, (LONG)TRUE); // accept edit
                    return TRUE;
                }
                default:
                    break;
            }
        }
        default:
            break;
    }

    return FALSE;
}

INT_PTR CALLBACK EditBoxCtl_SubclassProc(HWND hWndEditBox, UINT message,
                                         WPARAM wParam, LPARAM lParam) {
    switch(message) {
        HANDLE_MSG(hWndEditBox, WM_GETDLGCODE, EditBoxCtl_OnGetDlgCode);
        HANDLE_MSG(hWndEditBox, WM_KEYDOWN, EditBoxCtl_OnKey); // does not receive WM_KEYDOWN for ESC unless I handle WM_GETDLGCODE above
        default:
            break;
    }

    return CallWindowProc(TreeViewGlobals::g_wpOrigEditBoxProc, 
                          hWndEditBox, message, wParam, lParam);
}

UINT EditBoxCtl_OnGetDlgCode(HWND hWndEditBox, LPMSG lpmsg) {
    if(lpmsg) {
        if(lpmsg->message == WM_KEYDOWN && lpmsg->wParam == VK_ESCAPE) {
            return DLGC_WANTMESSAGE;
        }
    }

    return 0;
}

void EditBoxCtl_OnKey(HWND hWndEditBox, UINT vk, BOOL fDown, 
                      int cRepeat, UINT flags) {
    switch(vk) {
        case VK_ESCAPE:
                Beep(4000, 150); // never beeps
            break;
        default:
            break;
    }
}

---

P.S. I noticed that when I remove WM_GETDLGCODE handler in EditBoxCtl_SubclassProc(), it becomes possible to type in the edit box again, but then I can't trap WM_KEYDOWN for ESC key from that procedure.

Answer 1

Below is the solution that I found. The trick seems to be calling the original control proc with WM_GETDLGCODE intercepted in subclass proc, storing the return value and then returning it with DLGC_WANTALLKEYS or DLGC_WANTMESSAGE flag set to prevent system from further processing the keystroke.

The upside to this approach is that pressing ESC cancels editing and reverts the item label to its original text, and pressing ENTER while editing no longer just closes the dialog(which was another problem) without any additional code to handle those cases.

Here is the code that works:

INT_PTR CALLBACK EditBoxCtl_SubclassProc(HWND hWndEditBox, UINT message,
                                         WPARAM wParam, LPARAM lParam) {
    switch(message) {
        //HANDLE_MSG(hWndEditBox, WM_GETDLGCODE, EditBoxCtl_OnGetDlgCode);  // can't use this: need wParam and lParam for CallWindowProc()

        case WM_GETDLGCODE: {   
            INT_PTR ret = CallWindowProc(TreeViewGlobals::g_wpOrigEditBoxProc, 
                                         hWndEditBox, message, wParam, lParam);
            MSG* lpmsg = (MSG*)lParam;  
            if(lpmsg) {
                if(lpmsg->message == WM_KEYDOWN && 
                  (lpmsg->wParam == VK_ESCAPE || lpmsg->wParam == VK_RETURN) ) 
                {
                    return ret | DLGC_WANTALLKEYS;
                }
            }

            return ret;
        }

        default:
            break;
    }

    return CallWindowProc(TreeViewGlobals::g_wpOrigEditBoxProc, 
                          hWndEditBox, message, wParam, lParam);
}

Answer 2

The problem is that the modal dialog has its own message loop and its own translation with IsDialogMessage. Using the MFC I would say, just use PreTranslateMessage but this isn't available in plain WinApi. You don't have access to the internal message loop and the keyboard interface.

So the Escape key is handled inside the message loop. And causes a WM_COMMAND message with IDCANCEL to be sent. (See the MSDN specs about dialogs)

Maybe the easiest way is to interrcept the WM_COMMAND message sent to the dialog, check if who has the focus and if the inplace edit control has the focus you just set the focus back to the tree control and eat forget the IDCANCEL and don't close the dialog.

Answer 3

you need remember the tree-view hwnd when you receive TVN_BEGINLABELEDIT (in class member, associated with dialog) and zero it when you receive TVN_ENDLABELEDIT. when user press esc or enter in modal dialog box - you receive WM_COMMAND with IDCANCEL (on esc) or IDOK( on enter). you need check saved tree-view hwnd and if it not 0 - call TreeView_EndEditLabelNow

    switch (uMsg)
    {
    case WM_INITDIALOG:
        m_hwndTV = 0;
        break;
    case WM_NOTIFY:
        switch (reinterpret_cast<NMHDR*>(lParam)->code)
        {
        case TVN_BEGINLABELEDIT:
            m_hwndTV = reinterpret_cast<NMHDR*>(lParam)->hwndFrom;
            return TRUE;
        case TVN_ENDLABELEDIT:
            m_hwndTV = 0;
            //set the item's label to the edited text
            SetWindowLongPtrW(hwndDlg, DWLP_MSGRESULT, TRUE);
            return TRUE;
        }
        break;
    case WM_CLOSE:
        EndDialog(hwndDlg, 0);
        break;
    case WM_COMMAND:
        switch (wParam)
        {
        case IDCANCEL:
            if (m_hwndTV)
            {
                TreeView_EndEditLabelNow(m_hwndTV, TRUE);
            }
            else
            {
                EndDialog(hwndDlg, IDCANCEL);
            }
            break;
        case IDOK:
            if (m_hwndTV)
            {
                TreeView_EndEditLabelNow(m_hwndTV, FALSE);
            }
            else
            {
                EndDialog(hwndDlg, IDOK);
            }
            break;
        }
        break;
    }