Inherited a MFC based application that hosts both windowed and windowless ActiveX controls. In this application the user can design SCADA pages on which he/she is able to create and delete these AX controls dynamically. The controls are mostly simple widgets like buttons, readouts, lines, circles etc.
Since upgrading to a newer version of Visual Studio (2017) the program now crashes when deleting AX controls. Originally it was VC6.
More specifically, most crashes occur when you are attempting to delete a windowless AX control. If those AX controls are hosted in another host app say, Internet Explorer, then the crash does not occur.
It can also crash when the program is quitting, because then all the AX controls on the page are also deleted one by one.
Even more specifically, the order in which objects are created matters. If you delete a windowless object and the next object in focus is going to be a windowed object, you're safe: there is no crash. I think the "route events here" (focus) framework code then selects the windowed object as a valid target for window messages and everything will be fine from then on. If however the new object is also windowless, you are out of luck.
Note that we create all the controls using the WS_DISABLED flag, otherwise the button widget for instance will start reacting to mouse clicks when the user is sizing and moving around the widget.
When the GPF occurs after deleting a windowless AX control, the debugger breaks in and stops in the OLE container file occcont.cpp, function COleControlContainer::HandleWindowlessMessage. At that point it wants to call m_pSiteFocus->m_pWindowlessObject->OnWindowMessage.
This is in the context of the "container" window that hosts all the AX controls. A Window message is about to be routed to the currently focused AX control.
But here the incorrect destination is selected, because m_pWindowlessObject is 0xdddddddd (which means freed memory).
It looks to me like that site focus object should no longer be selected because all its content has been destroyed along with the deleted AX control!
Funny thing is that the delete action of the control itself does not cause any (direct) problems. Indirectly the "post delete" window messages now want to be delivered to an invalid OLE control container. And even when the program is shutting down, the program can still crash because of an earlier delete AX control action.
BTW: The AX control gets created with a CWnd::CreateControl and gets deleted again with a call to CWnd::DestroyWindow().
As a workaround, just after that destroy window call, I force the offending pointer to a null pointer:
COleControlContainer * pContainer = m_wndContainer.GetControlContainer();
const COleControlContainer * pFreedMemory(reinterpret_cast<COleControlContainer*>(0xdddddddd));
if (pContainer != nullptr && pContainer != pFreedMemory) {
// After we delete the control, the library wants to focus another control.
// But if there is no more control or the next control is also windowless/disabled
// it will bump into a stale pointer (0xdddddddd). The code does check for a null pointer,
// so we force a null pointer.
pContainer->m_pSiteFocus = 0;
}
Now the program does not crash anymore, but this is ugly!
So the question is: am I doing something wrong, am I facing a bug in the framework or perhaps even a bug in the windowless AX control?
Update #1:
Found this code fragment in MFC's occsite.cpp
BOOL COleControlSite::DestroyControl()
...
//VBBUG: VB controls will crash if IOleObject::Close is called on them
// when they have focus (and unfortunately, deactivating them does not
// always move the focus). To work around this problem, we always hide
// the control before closing it.
ShowWindow(SW_HIDE);
...
Apparently that's a workaround for a similar (if not the same) problem.
Update #2:
I found that for AX controls their destructor wasn't even called. Then indeed found a reference counted interface pointer that wasn't properly handled. Just as iinspectable predicted in the comments!
