P/Invoke PostMessage crashes once the window received message

Question

I'm creating a window via P/Invoke Win32 API in a C# console application on .NET core. Following is the core code.

class WindowContext
{
    public IWindow MainLoop(Action guiMethod)// this is called somewhere else
    {
        MSG msg = new MSG();
        while (msg.message != 0x12/*WM_QUIT*/)
        {
            if (PeekMessage(ref msg, IntPtr.Zero, 0, 0, 0x0001/*PM_REMOVE*/))
            {
                TranslateMessage(ref msg);
                DispatchMessage(ref msg);
            }
        }
    }

    private IntPtr WindowProc(IntPtr hWnd, uint msg, UIntPtr wParam, IntPtr lParam)
    {
        //....
    }

    public IWindow CreateWindow(Point position, Size size)// this is called to create a window
    {
        IntPtr hInstance = processHandle.DangerousGetHandle();
        string szAppName = "ImGuiApplication~";

        WNDCLASS wndclass;
        wndclass.style = 0x0002 /*CS_HREDRAW*/ | 0x0001/*CS_VREDRAW*/;
        wndclass.lpfnWndProc = WindowProc;

        // RegisterClass(ref wndclass);

        // CreateWindowEx(...)
        // ...
    }
}

But the program keeps crashing once I move the mouse onto the window.

The program '[18996] dotnet.exe' has exited with code -1073740771 (0xc000041d).

Finally I found out the crash occurred when the PeekMessage is called. But I couldn't tell why.

score 1 · Accepted Answer · edited May 23 '17 at 12:33

After searching and debugging for 3 hours, finally I found the cause.

The WinProc delegate instance is garbage collected. Then the native code will access a invalid function pointer.

I mean this one wndclass.lpfnWndProc = WindowProc;. The wndclass is a temporary object-a struct instance exactly-and will not exist on the stack when the program returns from CreateWindow. And after that, it is up to the CLR to determine whether to GC wndclass.lpfnWndProc.

So the solution is to make wndclass not a temporary object. For example,

class WindowContext
{
    WNDCLASS wndclass;
    public IWindow CreateWindow(Point position, Size size)// this is called to create a window
    {
        IntPtr hInstance = processHandle.DangerousGetHandle();
        string szAppName = "ImGuiApplication~";

        wndclass.style = 0x0002 /*CS_HREDRAW*/ | 0x0001/*CS_VREDRAW*/;
        wndclass.lpfnWndProc = WindowProc;
    }
}

Now the wndclass lives the same long with the WindowContext instance. Problem solved.

Some similar problems on SO:

https://stackoverflow.com/a/5007211/3427520

https://stackoverflow.com/a/1616718/3427520

P/Invoke PostMessage crashes once the window received message

1 Answers1

Linked