Select entity by mouse clicking with bevy & rapier

Question

So, I'm using Bevy (0.11) the Rapier (0.22) physics engine to have a bunch of blocks falling on each other, here is how I spawn the blocks:

commands.spawn((
    Collider::cuboid(dim, dim),
    MaterialMesh2dBundle {
        mesh: meshes. Add(shape::Box::new(dim2, dim2, dim2).into()).into(),
        material: materials.add(ColorMaterial::from(random_colour())),
        transform: Transform::from_xyz(
            left + (dim2 * i as f32) + 1.0,
            half_height - dim,
            0.0,
        ),
        ..default()
    },
    RigidBody::Dynamic,
    Restitution::coefficient(0.5),
    ColliderMassProperties::Mass(1000.0), // Increase the mass to stop the glitching (a bit)
    GravityScale(10.0),
));

I want to be able to interact with them via mouse click/touch. I'm trying to use the Point projection to see if the mouse is over something like so

fn mouse_button_events(
    mut mousebtn_evr: EventReader<MouseButtonInput>,
    q_windows: Query<&Window, With<PrimaryWindow>>,
    rapier_context: Res<RapierContext>,
) {
    use bevy::input::ButtonState;

    for ev in mousebtn_evr.iter() {
        match ev.state {
            ButtonState::Pressed => {
                println!("Mouse button press: {:?}", ev.button);

                let point = q_windows.single().cursor_position().unwrap();
                let filter = QueryFilter::default();

                rapier_context.intersections_with_point(point, filter, |entity| {
                    // Callback called on each collider with a shape containing the point.
                    println!("The entity {:?} contains the point.", entity);
                    // Return `false` instead if we want to stop searching for other colliders containing this point.
                    true
                });
            }
            ButtonState::Released => {
                println!("Mouse button release: {:?}", ev.button);
            }
        }
    }
}

But it's not printing the entity id when something is clicked. Are the screen coordinates the same as the Rapier space coordinates? What am I don't wrong here?

Does this answer your question? [How to read mouse motion in bevy?](https://stackoverflow.com/questions/66946558/how-to-read-mouse-motion-in-bevy) I’m a bit doubtful about this being a duplicate. — The Empty String Photographer, Aug 20 '23 at 18:52

score 1 · Answer 1 · answered Aug 23 '23 at 18:41

You need to convert the cursor position into a world position through the camera. The code you posted in the answer will only work if you don't move the camera from the initial position. A more general solution would be:

let (camera, camera_transform) = camera_q.single();
let Some(mouse_position) = windows.single().cursor_position()
    .and_then(|cursor| camera.viewport_to_world_2d(camera_transform, cursor))
else {
    return 
};

with

camera_q: Query<(&Camera, &GlobalTransform)>,

score 0 · Answer 2 · answered Aug 22 '23 at 08:40

Seems I was spot on about the different coordinate systems. Mouse events are in screen coordinates which have their origin in the top left while Rapier has it's origin in the center of the screen. So I just needed a conversion function...

fn screen_to_rapier_coords(point: Vec2, screen: Vec2) -> Vec2 {
    // Conver between screen coordinates with origin at the top left
    // to rapier coordinates with origin at the center
    let x = point.x - screen.x / 2.0;
    let y = screen.y / 2.0 - point.y;
    Vec2::new(x, y)
}

Select entity by mouse clicking with bevy & rapier

2 Answers2