Rust Win32 window, peek message disabling main loop exit

Question

I am new to Rust and i am learning it, however when i search for information around this issue, there is simply no information.

The problem exists in the main loop of the program for which the code looks like this (i will post the fill code for the 2 files at the end for people to read):

main loop function

    pub fn run(&mut self) -> isize {
        let mut msg = MSG {
            hwnd: null_mut(),
            lParam: 0,
            message: 0,
            pt: POINT {
                x: 0,
                y: 0
            },
            time: 0,
            wParam: 0,
        };

        loop {
            if unsafe { PeekMessageW(&mut msg, self.hwnd, 0, 0, PM_REMOVE) } != 0 {
                if msg.message == WM_QUIT {
                    break;
                }

                unsafe { TranslateMessage(&msg); }
                unsafe { DispatchMessageW(&msg); }
            } else {
                // TODO: Add stuff here
            }
        }
        0
    }

The desired behaviour is that when the PostQuitMessage(0) is send in the window_proc function the window is destroyed and the loop inside of run intercepts the WM_QUIT message and then break the loop.

The actual behaviour i am finding is that the window closes and is destroyed but the "main loop" in the run function never detects the WM_QUIT message and subsequentially never exits, leaving the window destroyed and the main loop still running.

win64.rs

#![cfg(windows)]

use std::error::Error;
use std::ptr::null_mut;
use std::mem::size_of;
use winapi::shared::minwindef::*;
use winapi::shared::windef::*;
use winapi::um::libloaderapi::{ GetModuleHandleW };
use winapi::um::wingdi::*;
use winapi::um::winuser::*;

fn to_wstring(s: &str) -> Vec<u16> {
    use std::os::windows::ffi::OsStrExt;
    std::ffi::OsStr::new(s).encode_wide().chain(std::iter::once(0)).collect()
}

pub extern "system" fn window_proc(hwnd: HWND, msg: UINT, wparam: WPARAM, lparam: LPARAM) -> LRESULT {
    match msg {
        WM_CLOSE => {
            unsafe { PostQuitMessage(0) };
            unsafe { DestroyWindow(hwnd) };
            0
        }
        WM_DESTROY => {
            unsafe { PostQuitMessage(0) };
            0
        }
        _ => return unsafe { DefWindowProcW(hwnd, msg, wparam, lparam) }
    }
}

pub struct Window {
    h_instance: HINSTANCE,
    hwnd: HWND,
    rect: RECT,
}

impl Window {    
    pub fn new(title: &str, width: u32, height: u32) -> Result<Window, Box<(dyn Error + 'static)>>  {
        let title = to_wstring(title);
        let class_name = to_wstring("_class64");

        let mut wnd = Window {
            h_instance: unsafe { GetModuleHandleW(null_mut()) },
            hwnd: null_mut(),
            rect: RECT {
                left: 0,
                top: 0,
                right: width as i32,
                bottom: height as i32,
            }
        };

        if wnd.h_instance == null_mut() {
            let err_str = "failed to obtain window instance";
            unsafe { MessageBoxW(null_mut(), to_wstring(err_str).as_ptr(), to_wstring("Error").as_ptr(), MB_ICONERROR | MB_OK); }
            return Err(err_str.into());
        }

        let wnd_class_ex = WNDCLASSEXW {
            cbClsExtra: 0,
            cbSize: size_of::<WNDCLASSEXW>() as u32,
            cbWndExtra: 0,
            hbrBackground: unsafe { GetStockObject(BLACK_BRUSH as i32) as HBRUSH}, 
            hCursor: unsafe { LoadCursorW(null_mut(), IDC_ARROW) },
            hIcon: unsafe { LoadIconW(null_mut(), IDI_APPLICATION) },
            hIconSm: unsafe { LoadIconW(null_mut(), IDI_APPLICATION) },
            hInstance: wnd.h_instance,
            lpfnWndProc: Some(window_proc),
            lpszClassName: class_name.as_ptr(),
            lpszMenuName: null_mut(),
            style: CS_OWNDC | CS_HREDRAW | CS_VREDRAW,
        };

        if unsafe { RegisterClassExW(&wnd_class_ex) } == 0 {
            let err_str = "failed to register window";
            unsafe { MessageBoxW(null_mut(), to_wstring(err_str).as_ptr(), to_wstring("Error").as_ptr(), MB_ICONERROR | MB_OK); }
            return Err(err_str.into()); 
        }

        wnd.hwnd = unsafe { CreateWindowExW(
            0,
            class_name.as_ptr(),
            title.as_ptr(),
            WS_OVERLAPPEDWINDOW | WS_VISIBLE, 
            CW_USEDEFAULT, CW_USEDEFAULT,
            wnd.rect.right, wnd.rect.bottom, 
            null_mut(), null_mut(), wnd.h_instance, null_mut())
        };

        if wnd.hwnd == null_mut() {
            let err_str = "failed to create window";
            unsafe { MessageBoxW(null_mut(), to_wstring(err_str).as_ptr(), to_wstring("Error").as_ptr(), MB_ICONERROR | MB_OK); }
            return Err(err_str.into());
        }

        unsafe { ShowWindow(wnd.hwnd, SW_SHOW); }
        unsafe { UpdateWindow(wnd.hwnd); }

        Ok(wnd)
    }

    pub fn run(&mut self) -> isize {
        let mut msg = MSG {
            hwnd: null_mut(),
            lParam: 0,
            message: 0,
            pt: POINT {
                x: 0,
                y: 0
            },
            time: 0,
            wParam: 0,
        };

        loop {
            if unsafe { PeekMessageW(&mut msg, self.hwnd, 0, 0, PM_REMOVE) } != 0 {
                if msg.message == WM_QUIT {
                    break;
                }

                unsafe { TranslateMessage(&msg); }
                unsafe { DispatchMessageW(&msg); }
            } else {
                // TODO: Add stuff here
            }
        }
        0
    }
}

main.rs

mod win64;

fn main() {
    println!("running create_window_win32api");

    let mut wnd = win64::Window::new("test window", 1280, 720).expect("failed to create window");
    wnd.run();
}

Why are you running a busy loop? Why you don't use a standard message loop with `GetMessage(&msg, 0, 0, 0)`? The answer to the question though, I expect, is that the WM_QUIT message isn't sent to your specific window so you won't retrieve it. In fact, your message loop will fail to process messages that aren't sent to `self.hwnd`. Shouldn't you be passing `0` instead? — David Heffernan, Mar 13 '23 at 14:57
@DavidHeffernan `GetMessage` is blocking and awaits for a message, `PeekMessage` is none blocking and continues to the loop and any code within it. You can find more information about both functions at MSDN — chloedev, Apr 27 '23 at 02:03
Obviously I understand this, which is why I was able to question your choice of a busy loop. Windows programs generally yield the thread when they are iidle. Your program won't. Why did you choose that? Is that really what you intend? — David Heffernan, Apr 27 '23 at 04:25

score 2 · Accepted Answer · answered Mar 13 '23 at 15:56

2

The bug is here¹:

PeekMessageW(&mut msg, self.hwnd, 0, 0, PM_REMOVE)

This call retrieves any pending messages posted to the target window. PostQuitMessage, however, posts a thread message, that's not associated with any window, and consequently doesn't match the window filter.

To fix the issue use the following instead:

PeekMessageW(&mut msg, null_mut(), 0, 0, PM_REMOVE)

This will retrieve any message posted to any window owned by the calling thread and thread messages.

That aside, if you want to write Windows code using Rust, ditch the winapi crate and go with the windows crate instead. The latter is actively developed and maintained (0.46.0 shipped an hour ago).

¹ See The dangers of filtering window messages

answered Mar 13 '23 at 15:56

IInspectable

46,945
8
85
181

Thank you so much for your response, you are correct the reason was the `hwnd` parameter. Once i changed to `null_mut()` as you said the code compiled and worked correctly, and strictly as expected. I may have to use the other crate you suggested, either that or return back to C++. But i want to remain relevant with the new and safer technology. Thank you for your advice. – chloedev Mar 13 '23 at 16:15
@chloedev Just to put the *"safer technology"* into perspective: `unsafe` Rust is no safer than C (a tiny little, maybe), and with equally complex rules that the compiler won't help you with. Note in particular that a single bug in an `unsafe` block has the capacity to invalidate all language guarantees that would otherwise apply to safe Rust. Tread carefully when in an `unsafe` block. – IInspectable Mar 16 '23 at 07:50

Rust Win32 window, peek message disabling main loop exit

1 Answers1