2D Screen coordinate to 3D position Directx 9 / Box Select

Question

I am trying to implement box select in a 3d world. Basically, click, hold mouse, and then unpress mouse, get a box, and then box select. To start, I'm trying to figure out how to get the coordinates of the clicks in 3d.

I have raypicking, and that is not getting the right coordinate (gets origin and direction). It keeps returning the same origin no matter what X/Y for screen is (although the direction is different).

I've also tried:

D3DXVECTOR3 ori = D3DXVECTOR3(sx, sy, 0.0f);
D3DXVECTOR3 out;
D3DXVec3Unproject(&out, &ori, &viewPort, &projectionMat, &viewMat, &worldMat);

And it gets the same thing, the coordinates are very close to each other no matter what coordinates (and are wrong). It's almost like returning the eye, instead of the actual world coordinate.

How do I turn 2d Screen coordinates into 3d using directx 9c?

The return values are correct. For the origin you will get points very near to the eye (in fact points on the near plane). You probably want to go the other way aroung. Project world points to the screen. Because a box on the screen is not a box in world space anymore. — Nico Schertler, Oct 21 '13 at 12:58
http://stackoverflow.com/questions/19150215/3d-screenspace-raycasting-picking-directx9/19153090#19153090 Should help to build ray — mrvux, Oct 21 '13 at 17:02
Hmm, any idea how to use that to do a box select (dragging mouse and pretending it's a box, and selecting everything inside? It's not working because the points are pretty much identical when I get them back. — Mary Ellen Bench, Oct 22 '13 at 04:15
If D3DXVec3Unproject doesn't give you the expected results, there is something wrong with your viewport or the matrices. Show us the matrices or how you create them. — cdoubleplusgood, Oct 22 '13 at 07:47
What seems to be happening is rays are identical to the eye, but I do picking fine (but of course the center is the eye, but I use the ray). But that doesn't seem to help for box select, since it's like one origin point. — Mary Ellen Bench, Oct 24 '13 at 04:57
Maybe I need to rephrase the question. As you move the cursor, if I wanted to draw a little cube in 3d, where I was hovering, how would I find the hover position? The ray/origin approach always gets the origin at exactly the eye, so that won't help? — Mary Ellen Bench, Oct 31 '13 at 05:44

zdd · Answer 1 · 2016-03-04T06:49:07.263

This is called picking in Direct3D, to select a model in 3D space, you mainly need 3 steps:

Generate the picking ray
Transform the picking ray and the model you want to pick in the same space
Do a intersection test of the picking ray and the model

Generate the picking ray

When we click the mouse on the screen(say the point is s on the screen), the model is selected when the box project on the area surround the point s on the projection window. so, in order to generate the picking ray with the given screen coordinates (x, y), first we need to transform (x,y) to the projection window, this is can be done by the invert process of viewport transformation. another thing is the point on the projection window was scaled by the project matrix, so we should divide it by the scale factors. in DirectX, the camera always place at the origin, so the picking ray starts from the origin, and projection window is the near clip plane(z=1).this is what the code has done below.

Ray CalcPickingRay(LPDIRECT3DDEVICE9 Device, int screen_x, int screen_y) 
{
     float px = 0.0f;
     float py = 0.0f;

     // Get viewport
     D3DVIEWPORT9 vp;
     Device->GetViewport(&vp);

     // Get Projection matrix
     D3DXMATRIX proj;
     Device->GetTransform(D3DTS_PROJECTION, &proj);

     px = ((( 2.0f * screen_x) / vp.Width)  - 1.0f) / proj(0, 0);
     py = (((-2.0f * screen_y) / vp.Height) + 1.0f) / proj(1, 1);

     Ray ray;
     ray._origin    = D3DXVECTOR3(0.0f, 0.0f, 0.0f);
     ray._direction = D3DXVECTOR3(px, py, 1.0f);

     return ray;
}

Transform the picking ray and model into the same space.

We always obtain this by transform the picking ray to world space, simply get the invert of your view matrix, then apply the invert matrix to your pickig ray. // transform the ray from view space to world space

void TransformRay(Ray* ray, D3DXMATRIX* invertViewMatrix)
{
    // transform the ray's origin, w = 1.
    D3DXVec3TransformCoord(
        &ray->_origin,
        &ray->_origin,
        invertViewMatrix);

    // transform the ray's direction, w = 0.
    D3DXVec3TransformNormal(
        &ray->_direction,
        &ray->_direction,
        invertViewMatrix);

    // normalize the direction
    D3DXVec3Normalize(&ray->_direction, &ray->_direction);
}

Do intersection test

If everything above is well, you can do the intersection test now. this is a ray-box intersection, so you can use function D3DXboxBoundProbe. you can change the visual mode of you box to see if the picking was really work, for example, set the fill mode to solid or wire-frame if D3DXboxBoundProbe return TRUE.

You can perform the picking in response of WM_LBUTTONDOWN.

case WM_LBUTTONDOWN:
{
    // Get screen point
    int iMouseX = (short)LOWORD(lParam) ;
    int iMouseY = (short)HIWORD(lParam) ;

    // Calculate the picking ray
    Ray ray = CalcPickingRay(g_pd3dDevice, iMouseX, iMouseY) ;

    // transform the ray from view space to world space
    // get view matrix
    D3DXMATRIX view;
    g_pd3dDevice->GetTransform(D3DTS_VIEW, &view);

    // inverse it
    D3DXMATRIX viewInverse;
    D3DXMatrixInverse(&viewInverse, 0, &view);

    // apply on the ray
    TransformRay(&ray, &viewInverse) ;

    // collision detection
    D3DXVECTOR3 v(0.0f, 0.0f, 0.0f);
    if(D3DXSphereBoundProbe(box.minPoint, box.maxPoint &ray._origin, &ray._direction))
    {
        g_pd3dDevice->SetRenderState(D3DRS_FILLMODE, D3DFILL_SOLID);
    }
    break ;
}

I appreciate the effort spent trying to help, however, picking is working, but doing a "box select", is not. Because whenever I try and convert 2d coordinates to 3d, such as for picking as you have, I only get the origin of the eye for both start and end (with different ray directions). Therefore I get no Y dimension, since both start and end box select click is the same coordinate, so you can't really do a bounding select. — Mary Ellen Bench, Oct 26 '13 at 15:38
if picking is work, why not box select? what do you mean by select? what do you want to do after you get the box? — zdd, Oct 27 '13 at 03:45
I want to select all the triangles enclosed in the "boxcast". Picking only gets those in the direct line. I'd click, hold mouse to second position, then get all triangles selected in the box (much like modellers do) — Mary Ellen Bench, Oct 27 '13 at 15:25

score 0 · Accepted Answer · answered Jul 24 '15 at 18:05

0

It turns out, I was handling the problem the wrong/opposite way. Turning 2D to 3D didn't make sense in the end. But as it turns out, converting the vertices from 3D to 2D, then seeing if inside the 2D box was the right answer!

answered Jul 24 '15 at 18:05

Mary Ellen Bench

589
1
8
28

2D Screen coordinate to 3D position Directx 9 / Box Select

2 Answers2

Generate the picking ray

Transform the picking ray and model into the same space.

Do intersection test