How do you prevent the VS Shell from getting priority on key/command bindings in a VSPackage ToolWindow?

Question

I have wpf a user control that accepts input gestures to perform various commands. In the example image below, the user can press Ctrl+N to execute the New command to create a new item in the TreeView.

enter image description here

When this user control is hosted within a WPF application, the user can create a new item using Ctrl+N when the focus is on the TreeView, and when the user presses Ctrl+N when the focus is elsewhere in the application, the application level New command is executed.

When this user control is hosted within a VSPackage ToolWindow, the VS Shell level New command to create a new file is executed when the user presses Ctrl+N, regardless of focus on the TreeView.

How do you prevent the VS Shell from getting priority on key/command bindings?

score 0 · Accepted Answer · edited Aug 02 '18 at 08:10

Some approaches were suggested by moderator Ryan on MSDN. The VS Shell is passing Winform messages to the ToolWindow, so the current approach is to override PreProcessMessage in the ToolWindow, handle the key combinations such as Ctrl+N that would map to WPF commands, and then translate those messages and pass to ComponentDispatcher.RaiseThreadMessage.

This approach is error prone in that the mapping to WPF commands could get out of sync, plus it doesn't incorporate whether or not the WPF command can execute. If anyone has a more ideal approach, please let me know, or if I find one, will post.

Perhaps there is a way for the WPF control to pass back whether the command was executed, and then only if the command was executed would the message be handled. Then, all this mapping business could be avoided.

    private bool isControlKeyDepressed = false;
    private bool isShifKeyDepressed = false;
    private bool isOtherKeyDepressed = false;
    private bool isCommandCombinationDepressed = false;

    protected override bool PreProcessMessage(ref Message msg)
    {
        // trap keyboard messages if window has focus
        if (msg.Msg == 256)
        {
            if (msg.WParam == (IntPtr)17)
            {
                isControlKeyDepressed = true;
                isOtherKeyDepressed = false;
            }
            else if (msg.WParam == (IntPtr)16)
            {
                isShifKeyDepressed = true;
                isOtherKeyDepressed = false;
            }
            else
            {
                if (isOtherKeyDepressed == true)
                {
                    isControlKeyDepressed = false;
                    isShifKeyDepressed = false;
                }
                isOtherKeyDepressed = true;
                if (isControlKeyDepressed == true)
                {
                    if (isShifKeyDepressed == true)
                    {
                        switch (msg.WParam.ToInt64())
                        {
                            case 65: // Ctrl+Shift+A command
                            case 67: // Ctrl+Shift+C command
                            case 78: // Ctrl+Shift+N command
                            case 79: // Ctrl+Shift+O command
                            case 83: // Ctrl+Shift+S command
                            case 85: // Ctrl+Shift+U command
                            case 88: // Ctrl+Shift+X command
                                isCommandCombinationDepressed = true;
                                break;
                            default:
                                isCommandCombinationDepressed = false;
                                break;
                        }
                    }
                    else
                    {
                        switch (msg.WParam.ToInt64())
                        {
                            case 69: // Ctrl+E command
                            case 78: // Ctrl+N command
                            case 79: // Ctrl+O command
                            case 83: // Ctrl+S command
                                isCommandCombinationDepressed = true;
                                break;
                            default:
                                isCommandCombinationDepressed = false;
                                break;
                        }
                    }
                }
                else
                {
                    isCommandCombinationDepressed = false;
                }
            }

            if (isCommandCombinationDepressed == true)
            {
                // send translated message via component dispatcher
                MSG dispatchMsg = new MSG();
                dispatchMsg.hwnd = msg.HWnd;
                dispatchMsg.lParam = msg.LParam;
                dispatchMsg.wParam = msg.WParam;
                dispatchMsg.message = msg.Msg;
                ComponentDispatcher.RaiseThreadMessage(ref dispatchMsg);
                msg.Result = (IntPtr)1;
                return true;
            }
        }
        return base.PreProcessMessage(ref msg);
    }

`ComponentDispatcher.RaiseThreadMessage` returns a boolean value whose purpose is to indicate whether the message has been handled. Is that the notification you're looking for? You can opt to listen for those thread messages by subscribing to Filter and PreProcess on the ComponentDispatcher itself. Perhaps the ThreadFilterMessage event is where you determine whether an accelerator exists. — Greg D, Aug 18 '11 at 18:05
+1 Thanks Greg. I can indeed use the boolean return value to determine whether the message is handled and avoid the filtering logic and all of the commands are processed in the WPF windows. I'm a bit concerned about potential performance, as each call to RaiseThreadMessage requires creating an MSG instance from a Message instance, which is a whole lot of messages without any filtering. Could you expand on use of ThreadFilterMessage and ThreadPreprocessMessage events (I'm getting stack overflows calling ComponentDispatcher from there)? Post your response as an answer and I'll accept it. — Dave Clemmer, Aug 18 '11 at 20:15
I don't have a good answer, unfortunately. :) I'm working with `ComponentDispatcher` in a Win32-WPF interop scenario, so I can't address your question directly. One thing to note is that `ComponentDispatcher.RaiseThreadMessage` should _only_ be passed keyboard messages, per the documentation on `ComponentDispatcher`. E.g., "The owner calls RaiseThreadMessage for every keyboard message." Pretty much everything I know comes directly from the docs there. — Greg D, Aug 19 '11 at 13:29
`ThreadFilterMessage` and `ThreadPreprocessMessage` are fired as a result of calling `ComponentDispatcher.RaiseThreadMessage`. In fact, that's pretty much all `RaiseThreadMessage` does! If you call RaiseThreadMessage from a handler on those, you'll get a stackoverflow, yeah. — Greg D, Aug 19 '11 at 13:30

How do you prevent the VS Shell from getting priority on key/command bindings in a VSPackage ToolWindow?

1 Answers1