1

I need to convert a 2D mouse coordinate to a 3D world coordinate based on a specific depth (it's for raycasting). I'm not directly using DirectX in C++, I'm using a proprietary language, but if you give me an answer in C++ or pseudocode (C++ preferred) I can convert it.

I have access to the world matrix, view matrix and projection matrix and a variety of matrix manipulation functions.

If it's necessary to multiply a vector4 by a matrix4, the only function I have available that takes both a vector4 and a matrix4 is transformVector4(vector4Source,matrix4Source). I'm not sure which order it multiplies them in, if that matters.

Any help would be much appreciated :) I've been working on this for hours and I just don't get 3D maths...

Clonkex
  • 3,373
  • 7
  • 38
  • 55
  • calculate the ray on eye coordinate for your camera first(using some trigs) then transform that ray into world coordinate by multiplying with the inverse of modelview matrix of the camera. – yngccc Oct 03 '13 at 03:53
  • @yngum: It's easier to transform the target object coordinates into screen space (which is already done for rendering purposes anyway), and do hit-testing in 2D. – Ben Voigt Oct 03 '13 at 04:09
  • @BenVoigt yes I imagine that works too. op is asking for world coordinate so I thought of the straight forward way that myself have been doing. – yngccc Oct 03 '13 at 04:22
  • I don't have the faintest clue how to do the first comment-answer. Could you elaborate a bit more? Be specific? I don't know the target object coordinates (that's what I'm trying to calculate) and I have no access to low-level rendering stuff. Also, I need to have a start point (just the camera's position) and an end point and it must be a 3D raycast. I have tools for the raycasting itself, just not for calculating where the mouse cursor is in 3D. @yngum has the right idea, just not explained clearly. Thanks anyway :) – Clonkex Oct 03 '13 at 04:29
  • @Clonkex The answer is little long because I am not sure about your understanding of 3d transformation. You should google a bit, this is a frequent question and there should be plenty of tutorials. like [this one](http://www.opengl.org/archives/resources/faq/technical/selection.htm#sele0010) – yngccc Oct 03 '13 at 04:52
  • @yngum I barely understand matrices. All I wanted was some C++ or pseudocode. I have Googled it extensively but all of the tutorials and answers so far have been specific to OpenGL (most of them), not worked for me or been incomplete answers. [This page](http://gamedev.stackexchange.com/questions/6940/3d-ray-casting-picking) is the closest I gotten to a good answer, but it has some errors in the code and I'm not sure how it's supposed to look. Plus, why would I need a ModelView matrix for a camera? Why not just a View matrix? Is there even a difference? – Clonkex Oct 03 '13 at 05:11
  • @Clonkex the model matrix is for when your camera is placed not at the origin but otherwise unnecessary. Your link is basically how it work and is simpler. – yngccc Oct 03 '13 at 05:28
  • @Clonkex check out [this tutorial](http://www.bfilipek.com/2012/06/select-mouse-opengl.html) – yngccc Oct 03 '13 at 05:41
  • @yngum That's what I mean by OpenGL-specific. I can't use that tutorial because the most important bit, converting 2d to 3d, is handled by an OpenGL-specific function, gluUnProject. I'll do some more experimenting with modelview matrix now, but I doubt that I will succeed with anything... Remember, my link shows a broken method of calculating the 2d-3d conversion and I don't know what to do to make it correct. – Clonkex Oct 03 '13 at 06:12

1 Answers1

5

To convert you mouse to ray, you do this process:

Convert your mouse coordinates from pixel coordinate to -1/1 coordinates (-1,-1 being bottom left).

ray origin will be (vector at near plane)

vector3 origin = vector3(mousex,mousey,0);

ray end is (vector at far plane)

vector3 far = vector3(mousex,mousey,1);

now you need to apply transform to those vectors, first you need to create transformation matrix for it:

matrix4x4 inverseviewproj = invertmatrix(view * proj)

Apply this transformation to both vectors:

vector3 rayorigin = transform(origin, inverseviewproj);
vector3 rayend = transform(far, inverseviewproj);

Your ray direction is :

vector3 raydirection = normalize(rayend-rayorigin);

That's about it, now you can use raycast functions.

In case you only have access to vector4 to transform vector by a matrix, the w component needs to be 1 in eg:

vector4 origin = vector3(mousex,mousey,0,1);
vector4 far = vector3(mousex,mousey,1,1);

to extract direction make sure to first convert your vector4 into vector3 (since it will affect normalization otherwise).

mrvux
  • 8,523
  • 1
  • 27
  • 61