How can I make a click-through overlay in Rust that still captures input?

Question

I already have code that creates a transparent window and draws a square on it using winit and pixels, but I can't make it click-through, that is, let the user interact with what is behind the overlay window, while still letting the app capture input. Here's a minimal example of my code:

use pixels::{wgpu::Color, Pixels, SurfaceTexture};
use winit::{
    event::{DeviceEvent, ElementState, Event, KeyboardInput, VirtualKeyCode, WindowEvent},
    event_loop::EventLoop,
    platform::windows::{WindowExtWindows, HWND},
    window::{WindowBuilder, WindowLevel},
};

fn main() {
    let event_loop = EventLoop::new();
    let window = WindowBuilder::new()
        .with_fullscreen(Some(winit::window::Fullscreen::Borderless(None)))
        .with_transparent(true)
        .build(&event_loop)
        .unwrap();

    window.set_window_level(WindowLevel::AlwaysOnTop);
    window.set_cursor_hittest(false).unwrap();

    let window_size = window.inner_size();
    let surface = SurfaceTexture::new(window_size.width, window_size.height, &window);

    let mut pixels = Pixels::new(window_size.width, window_size.height, surface).unwrap();
    pixels.set_clear_color(Color::TRANSPARENT);

    event_loop.run(move |event, _, control_flow| {
        control_flow.set_wait();

        match event {
            Event::WindowEvent {
                event: window_event,
                ..
            } => match window_event {
                WindowEvent::KeyboardInput {
                    input:
                        KeyboardInput {
                            virtual_keycode: Some(VirtualKeyCode::Space),
                            state: ElementState::Pressed,
                            ..
                        },
                    ..
                } => {
                    println!("Input from window event");
                }

                WindowEvent::CloseRequested => control_flow.set_exit(),

                _ => (),
            },

            Event::DeviceEvent {
                event:
                    DeviceEvent::Key(KeyboardInput {
                        virtual_keycode: Some(VirtualKeyCode::Space),
                        state: ElementState::Pressed,
                        ..
                    }),
                ..
            } => {
                println!("Input from device event");
            }

            Event::RedrawRequested(_) => {
                pixels.render().unwrap();
            }

            _ => (),
        }
    });
}

I thought that Event::DeviceEvent would work because it seemed like it wasn't restricted to a specific window, but it is. In every scenario I've tried, both or none of the println!()s were called. Do I need another crate for that?

What do you need this for? It sounds suspiciously like the mouse version of a keylogger. — Finomnis, Mar 03 '23 at 20:39
@Finomnis Is that why I found it so difficult to find that information online? A friend of mine said it'd be fun to have a program that replicated the League of Legends ping as an overlay. I don't even play the game, but I thought it was a fun way to start learning the language. [My first question on Stack Overflow](https://stackoverflow.com/questions/75505017/how-can-i-make-rust-with-the-rodio-crate-load-multiple-sources-in-a-vec-so-i) was a few days ago about playing sound. It's the same project. The sound will be the ping sound. — Felipe, Mar 04 '23 at 00:18
It seems that the [device-query](https://crates.io/crates/device_query) can solve the problem. I don't have access to my computer right now, so I'll try it later. But I'd still prefer to do it in winit if possible. — Felipe, Mar 04 '23 at 01:06
@Finomnis I respect your suspicion because its true my problem is similar to a keylogger. However, first of all, because of your suspicion I searched for it, and there are simple keyloggers in the top results. Why would I do it myself in Rust? Trying to prevent people from learning how to do malicious stuff on the internet would also prevent self-taught programmers from learning advanced stuff that *could potentally* harm others. That includes most of the things that are not interacting with the terminal. Now let's get extreme: think about ethical hacking. How would that be taught? — Felipe, Mar 04 '23 at 18:59
My opinion, though. I'm sure that's not a new discussion. But the truth is that, in programming, you can always get to a malicious result through non-malicious stuff if you really want to. I am new to Rust and creating and handling windows, so I took some time to find my answer because I was not very open-minded about searching for a non-window approach, but it is now solved. — Felipe, Mar 04 '23 at 19:03

score 0 · Accepted Answer · answered Mar 04 '23 at 18:35

The crate device_query can solve the problem. It doesn't even need a window to function, as the input is queried on demand by calling DeviceState::get_keys() and DeviceState::mouse() on a DeviceState instance for keyboard and mouse respectively.

use device_query::{DeviceState, DeviceQuery};

// Cheaply creates an empty DeviceState
let device_state = DeviceState::new();

// Those methods query the input. They are individualy lazily queried.
let keys = device_state.get_keys();
let mouse = device_state.get_mouse();

let is_alt_pressed = keys.contains(&Keycode::LAlt);
let is_m1_pressed = mouse.button_pressed[1]; // It starts at [1] for M1. [0] has no meaning.

The above code can capture input without window focus and even without a window at all. In the code provided in the question, it should be put inside the MainEventsCleared event. It is also required to replace control_flow.set_wait() in the first line inside the event_loop.run() closure with control_flow.set_poll(), so that MainEventsCleared will always run, even without new events.

score 0 · Answer 2 · answered Mar 12 '23 at 07:05

0

It looks like window.set_cursor_hittest(false) will work although this is only supported on certain platforms.

answered Mar 12 '23 at 07:05

Bigbadboybob

928
2
11
18

That does allow me to click through the window, but it does not let me read the input. That part was already done in the code of the question, but with an ugly call to unsafe code, so thanks for the alternative method. – Felipe Mar 14 '23 at 01:51
I will edit the question so that method is used intead of `SetWindowLongPtrW()` – Felipe Mar 14 '23 at 01:57

How can I make a click-through overlay in Rust that still captures input?

2 Answers2