SetFocus to parent window, then back to child window

Question

I'm trying to implement a VST using VSTGUI 4.0. My DAW uses WM_KEYDOWN and WM_KEYUP messages to send midi notes based on keyboard presses, so that you can play VST's with the keyboard. The problem is, that VSTGUI does this when it receives a WM_LBUTTONDOWN message:

win32Frame->prevFocus = SetFocus (win32Frame->getPlatformWindow ());

CPoint where ((CCoord)((int)(short)LOWORD(lParam)), (CCoord)((int)(short)HIWORD(lParam)));
if (pFrame->platformOnMouseDown (where, buttons) == kMouseEventHandled)
    SetCapture (win32Frame->getPlatformWindow ());
return 0;

This steals focus from the DAW, and doesn't allow it to process key presses. VSTGUI needs the window focus in order to handle WM_MOUSEWHEEL and WM_KEYUP/WM_KEYDOWN events for tweaking controls. But when you're tweaking controls in the VST you obviously want to be able to play notes with the keyboard to see what they sound like, so both functionalities are important.

The only way I could figure to solve the problem was to SetFocus() to the parent window, send WM_KEYUP/WM_KEYDOWN messages back up to it using SendMessage(), then SetFocus() back to the VST window:

case WM_KEYDOWN:
    ...code to handle modifiers like shift, ctrl, etc...
    else
    {
        SetFocus(win32Frame->prevFocus);
        SendMessage(win32Frame->prevFocus, message, wParam, lParam);
        SetFocus(win32Frame->getPlatformWindow ());
    }

This works perfectly, until you click the VST and press a key at the same time, at which point undefined stuff happens (Freezes the DAW, crashes the DAW, stack overflow, etc).

So obviously I'm taking the wrong approach with this. I feel like I need to PostMessage() and wait for a callback before I return focus to the VST or something like that. Even that sounds kinda messed up though, so what exactly is the right way to deal with a problem like this?

Keep in mind I don't have access to the code of the DAW window that's passing the messages down, so I can't implement a custom message or anything like that.

Wow, that is some truly ugly code. Not sure if they could get more casts in there if they were getting paid for each one. If that's not a code smell, I don't know what is. Anyway, what I don't understand is why you need to capture the mouse. Isn't having the input focus good enough? — Cody Gray - on strike, Jul 06 '13 at 09:03
I need to give the input focus to the DAW so it can send note events via keyboard to play the VST. It's really weird, I kinda need the DAW and the VST to have keyboard focus at the same time. Even if I could just give the VST mouse wheel focus that would be enough, but it seems keyboard and mousewheel focus are one and the same. I can't write code to manually play midi notes in the VST either, because there are DAW specific things like octave I need to account for. Oh and yeah the code is absolutely atrocious haha, VST SDK and VSTGUI are probably the worst codebases I've ever had to work with. — superjoebob, Jul 06 '13 at 09:08
It appears that you're confusing input focus and capturing the mouse. The first snippet of code captures the mouse, which is what's breaking keyboard accelerators and mnemonics. That's a similar but distinct problem from having the input focus. Although it occurs to me that if the problem is getting an unfocused window to receive scroll messages, you could just handle them on the focused window and forward them onto the unfocused window. The window doesn't actually need to have the input focus to scroll, it just needs the focus to receive the messages without special intervention. — Cody Gray - on strike, Jul 06 '13 at 09:12
The problem with that is I don't have access to the parent window code, it belongs to the DAW (FL Studio). So I can't really forward messages from it. That's why I was trying to pass the message up instead, which really does feel like breaking form in a trickle down system like windows messages. I took a look at global message hooks, but that seems kinda sketchy? I'm not sure how that sort of thing handles when the window is out of focus and behind other windows. — superjoebob, Jul 06 '13 at 09:21
Processing keyboard messages when the window doesn't have the focus is entirely normal. The boilerplate example is Alt+F4, works regardless of focus. It is done inside the message loop, before the message is dispatched where you normally write TranslateMessage() and TranslateAccelerator(). If you can't change the DAW then you have a problem. PostMessage() could work somewhat but do it without the focus changes and by using a reliable window handle. The somewhat more likely source of the problem is that you don't implement the ActiveX contract properly. — Hans Passant, Jul 06 '13 at 09:36
Hans you might have just solved my problem, I'm gonna test it some more and confirm tomorrow! — superjoebob, Jul 06 '13 at 09:59
It works perfectly, all I did was replace my SetFocus/SendMessage/SetFocus chunk with a single PostMessage, and got my window handle with GetParent instead of storing it. Thanks a bunch Hans you nailed it. This is my first time actually posting on Stack Overflow so I'm not sure how selecting an answer works? — superjoebob, Jul 06 '13 at 22:27
They have to post it as an answer in order for you to accept it. Hans and I have posted comments, which can't be accepted. But it's perfectly fine to answer your own question, which you can then accept. — Cody Gray - on strike, Jul 07 '13 at 08:07
2.4, I feel like I can support more setups that way. It seems to me VST 3 isn't too terribly common yet? Or am I completely crazy? — superjoebob, Jul 12 '13 at 06:11

score 0 · Accepted Answer · answered Jul 07 '13 at 21:10

Big thanks to Hans Passant for solving this!

The crash issue was due to my window handle being unreliable. I stopped storing the window handle in a variable, and instead I got it on the spot using GetParent().

I also didn't need to set focus to the current window, all I needed to do was replace my SendMessage with a PostMessage. So the new code:

case WM_KEYDOWN:
    ...code to handle modifiers like shift, ctrl, etc...
    else
    {
        PostMessage(GetParent(win32Frame->getPlatformWindow ()), message, wParam, lParam);
    }

SetFocus to parent window, then back to child window

1 Answers1