Usable World Ray from 2D Mouse Coordinates?

Question

I'd like to construct a world ray given a mouse click, which can be used to run intersection checks.

The ray should be generated, given glm::vec2 mouse coordinate and glm::mat4 projection and view matrix's. Taking into account near and far planes. How can this be implemented in C?

Edit: I have constructed a mouse ray from mousePosition, view and projection. the coordinates it returns are close to what I'm after however mouseRay.z isn't behaving at all.

glm::vec2 mousePosition = { .x, .y };
glm::vec2 normalisedCoords = {
 -1.0f + 2.0f * position.x / width,
  1.0f + -2.0f * position.y / height
};
glm::vec4 clipCoords = { normalisedCoords.x, normalisedCoords.y, -1, 1 };
glm::mat4 invertedProjection = glm::inverse(projection);
glm::vec4 eyeCoords = invertedProjection * clipCoords;
glm::mat4 invertedView = glm::inverse(view);
glm::vec4 rayWorld = invertedView * eyeCoords;
glm::vec3 mouseRay = { rayWorld.x, rayWorld.y, rayWorld.z };
printf("x: %f, y: %f, z: %f\n\r", rayWorld.x, rayWorld.y, rayWorld.z);

Answer 1

Given a projection matrix: [P], and view matrix: [V], the transformation: [M] = [P][V] is applied to yield geometry in a homogeneous clip coordinate space (CCS). After projection, you have a point in normalized device coordate space (NDCS). That point is then mapped to your viewport, and a depth buffer - which is why it's sometimes referred to as a '3d viewport'.

Given the 2D mouse coordinates: (x, y), you need a linear transform to map (x, y) back to normalized coordinates: (x_ndc, y_ndc) using the viewport: {vx, vy, vw, vh} parameters. You can lookup how OpenGL maps to the '3D viewport', and apply the inverse of this transform to (x) and (y) coordinates.

This yields a (homogeneous) point on the picking ray: (x_ndc, y_ndc, -1, +1) ... it's a point on the W = - Z plane, which is what the 'near' plane was mapped to - don't get too bothered by notions of clipping (hyper) planes at this point!

Assuming the eye point: (ex, ey, ez, 1), is in world coordinate space (WCS), we want to find: inv([M]) = inv([V]) * inv([P]) to apply to: (x_ndc, y_ndc, -1, +1).

So the mouse point (x, y) represents: r1 = inv([M]) * (x_ndc, y_ndc, -1, +1) in world coordinate space. The origin of the ray is the eye point: r0 = (ex, ey, ez, +1). Finally, the ray direction: rt = r1 - r0 * r1.w ensures: rt.w = 0 - a vector, not a point.

You now have a ray in world coordinate space: r(t) = r0 + t.rt, t > 0

---

This does require a good understanding of matrix properties, homogeneous coordinates, etc. But you should be able to find some other GLM examples that construct a picking ray originating at the eye using these principles. You can always dig more deeply into the theory later, and satisfy yourself as to why this works.

Answer 2

Your view frustum has near and far planes(the far one is where the visibility gets cut off). A mouse click is two points:

1. one plane representing viewer's eye.
2. one in the far view frustum representing the flat horizon. The flat horizon is scaled 2d plane from the viewer's eye.

Therefore you can construct a 3d line segment by connecting the near and far segments. In notation it be (nearx,neary,nearz) and (farx,fary,farz) However using the observation about 2. you can say how you set up the frustrum that

farx = nearx - xscale fary = neary - yscale farz = nearz - zscale.

Note all scale variables are constants

Depending on your coordinate system and projection x and y could mean different things in your setup. It's best to just take your frustrum and measure out those values to know what goes where.

Ok from this you can plug in: (nearx,neary,nearz), (nearx-xscale,neary-yscale,nearz - zscale)

That's two coordinates but a ray is simple a vector which signifies magnitude and direction(and the line coming from one segment to the other). So todo that just apply subtraction [nearx - (nearx -xscale), neary - (neary-yscale), nearz -(nearz - zscale)]

Technically the ray can be two ways depending on which way you build it up either from the eye to the far away or faraway to the eye.

To run intersections you'll have to convert the ray from view space to world space.

You can google ray picking for your favorite api. The implementation will depend on the api you choose. But this is the linear algebra in a nutshell.

One more note. Depending on your use case it may not be desirable to have a ray mouse click. It's possible you want the planes perpendicular. For certain tile based problems this could be desirable. This can be done easy by setting them the same 2d sizes for certain axis.