DirectX Unproject troubles

Question

I have an orthographic projection and I try to unproject a point from screen space.

Following are the view and projection matrices:

var w2 = ScreenWidthInPixels/2;
var h2 = ScreenHeightInPixels/2;

view  = Matrix.LookAtLH(new Vector3(0, 0, -1), new Vector3(0, 0, 0), 
                        Vector3.UnitY);
proj  = Matrix.OrthoOffCenterLH(-w2, w2, -h2, h2, 0.1f, 10f);

Here is how I unproject a Point p, the point is given in screen pixels:

var m = Vector3.Unproject(p, 0, 0, ScreenWidthInPixels, ScreenHeightInPixels, 
                  0.1f, 10f, // znear and zfar
                  view *proj);

My code doesn't work, the matrix m contains only Nan. When I try to invert view * proj I get back a Matrix with only zeros.

So I suspect my problem has something to do with the orthographic projection matrix.

Here are my questions:

• Could the problem be caused by an underflow due to the large values in the OrthoOffCenterLH projection?
• What parameters do I have to pass for x,y,width,height in Unproject(...)?
• What significance has the minZ and maxZ parameter in Unproject(...)?
• Does it matter what I pass for p.Z in Unproject(...)?

Answer 1

Unfortunately, the SharpDX documentation is not online right now. So my explanations might be based on an older version I found the code of.

Could the problem be caused by an underflow due to the large values in the OrthoOffCenterLH projection?

What large values? I assume the screen width and height to be less than 2000. That is definitely not large. Don't worry about this.

What parameters do I have to pass for x,y,width,height in Unproject(...)?

These are the dimensions of the viewport. This is needed, because the view and projection matrix transforms a vector into clip space. This is usually [-1, 1] for x and y and [0, 1] for z. The position first needs to be translated from screen space into clip space. That's what those values are for.

What significance has the minZ and maxZ parameter in Unproject(...)?

This is how you encode the z-component of the input vector. It is also used for normalizing the coordinate to screen space. In general, you can leave it at 0 and 1.

Does it matter what I pass for p.Z in Unproject(...)?

Yes, of course. This parameter specifies the depth in the view frustum / cube. If you pass in minZ, you get the unprojected coordinates of the point on the front face. If you pass in maxZ, you get the position on the back face. Every value in between results in a coordinate in between. That's because a 2D point maps to an entire ray of 3D points.

Concluding, the Unproject method should work with an orthographic projection matrix. In your case, the resulting view*proj is just a simple combination of scaling and translations, which should be perfectly invertible. Are you sure that the resulting inverse contains only zeros? If so, then please add the original matrices to your question. Maybe there is something else going on.