1

I've been working on mouse picker on a OpenGL project and followed this guide to transform my 2D mouse position to a 3D vector on click.

The problem I have is in my calculations for picking the object/mesh. It seems to work properly if I am in the origin position of the camera, but as soon as I start to move around with the camera my calculated bounding sphere for the mesh gets out of position. I assume that I need to do some kind of matrix transformation to fix this problem but I can't figure it out. I do have access to the view matrix but I'm not sure If I should implement it in my calculations.

A video of my problem. It can be a bit hard to understand the problem with no click sounds. It gets more visually clear towards the end when I am able to select the object by clicking next to it.

Data I have for calculations:

  • The position of the camera (rayOrigin in calculations)
  • The mesh and all of its vertices and also the position of the mesh in world space.
  • The 3D vector for the mouse click calculated with the same method as the guide mentioned above.

The first thing I've done is to re-calculate all the objects vertices to fit the position of the mesh in world space. The position of the mesh should also be the center of the mesh (at least in the video I linked earlier).

vertecies[i] = vec3(worldMatrix * vec4(vertecies[i], 1));

In my second step I compare each vertex with the center position and calculate which vertex that has the largest magnitude from the center. This vectors lenght I use as radius for my bounding sphere.

My calculations for the bounding sphere ray test, rayDir is the mouse position vector, rayOrigin is the camera position:

vec3 oc = rayOrigin - center;
float doc = dot(oc , rayDir);

if (doc > 0 || (dot(doc, doc) < radiusSquared)) return false; // no hit

vec3 a = oc - doc * rayDir;
float aSquared = dot(a, a);

if (aSquared > radiusSquared) return false; // no hit

return true; // hit

My linear algebra isn't the best so I'm hoping you guys can help me out and tell me what I do wrong here.

Olivier Moindrot
  • 27,908
  • 11
  • 92
  • 91
Emil Gustafsson
  • 11
  • 4

1 Answers1

0

I've been looking at this problem, but referring to this webpage. I find your reference hard to follow from a developer's point of view, and your math is inconsistent with my reference. For example, your first condition should be: 

if (doc < 0) return false;

I'm not going to study your code to see if it's mathematically equivalent. But, I would suggest you start with the code towards the bottom of my reference. Also, verify center is the center of the sphere in the same coordinate system as your ray.

As a side note:

It always helps me to draw lines in the scene when I run into issues like these. In this case I would convert the ray to world space, and don't forget the camera eye (in world space) is extracted from the inverse view matrix. I prefer to pull it out of the inverse directly, so I don't have to question if I'm relying on stale cached data.

stephen
  • 1,039
  • 14
  • 31