0

I have my own unproject function for performing reverse projection of a screen point. The code is as follows (written in OpenTK):

public static Vector3 UnProject(Point screenLocation, float depth)
{
    int[] viewport = GetViewport();
    Vector4 pos = new Vector4();

    // Map x and y from window coordinates, map to range -1 to 1 
    pos.X = (screenLocation.X - viewport[0]) / (float)viewport[2] * 2.0f - 1.0f;
    pos.Y = 1 - (screenLocation.Y - viewport[1]) / (float)viewport[3] * 2.0f;
    pos.Z = depth * 2.0f - 1.0f;
    pos.W = 1.0f;

    Vector4 pos2 = Vector4.Transform(pos, Matrix4.Invert(GetModelViewMatrix() * GetProjectionMatrix()));
    Vector3 pos_out = new Vector3(pos2.X, pos2.Y, pos2.Z);

    return pos_out / pos2.W;
}

Basically, you'd provide the desired unprojection depth to my function, and it will give you the corresponding world coordinate of the screen point. Assuming that this works correctly (which I am 99% sure think it does), I'm having problems converting screen points to world coordinates. This unprojection works fine for picking: I'd call my unproject function twice (once with depth = 0 and another time with depth = 1) to convert the screen point to ray. I perform ray/triangle intersection to determine which object intersects with the ray and based on that I perform picking (which works very accurately).

For another operation (let's call it operation X), I only need to know the world coordinate of the screen point (assuming that the mouse cursor is over an object on the screen). For that, I am obtaining the depth under the cursor by using the glReadPixel function. The problem is that I feel the Z value obtained by reading the depth buffer is a little bit off. If I calculate the intersection with ray casting, I get accurate results, but that is not viable for operation X as operation X needs to be performed every time MouseMoved is triggered.

To demonstrate the lack of accuracy, here are the two numbers I obtained:
glReadPixel + Unprojection yields (0.886105343709181, 0.12422376198582, 0.998496665566841) as the world coordinate under the cursor.
Ray casting + intersection yields  (0.885407337013061, 0.124174778008613, 1) as the world coordinate under the cursor.

This 0.0015 error in the Z value is too much for operation X (as it is very sensitive to small numbers). Is there something wrong with glReadPixels that I should know about? Is this happening because glReadPixels is only capable of reading float values?

Maghoumi
  • 3,295
  • 3
  • 33
  • 49
  • 2
    I don't think that `glReadPixels` is to blame here. I think that the Z buffer precision is the issue. By default, you typically have a 24 bit fixed-point depth buffer. Maybe it hekps if you use a 32 bit floating point depth buffer, but you probably need an FBO for that. – derhass Jul 04 '15 at 20:57
  • @derhass Thanks! I didn't know about the precision issues. I checked and I was using 16 bit precision! (The default setting in OpenTK). I increased it to 24 and it seems to be much better now. I'm just curious, is my method of converting screen point to world point the correct "best practices" way of doing such thing? If not, what is the best option? Also please post your response as an answer so that I can accept it :) – Maghoumi Jul 04 '15 at 23:02
  • 1
    Well, it is unclear what the best way of doing this, especially as your "operation X" and its the precision requirements are left unknown. But going back from depth buffer to eye or world space is not uncommon, at least. Another alternative would be directly writing the eye or world space depth to some (FP32) render target during the rendering pass. – derhass Jul 04 '15 at 23:38

1 Answers1

1

I don't think that glReadPixels is to blame here. I think that the Z buffer precision is the issue. By default, you typically have a 24 bit fixed-point depth buffer. Maybe it helps if you use a 32 bit floating point depth buffer, but you probably need an FBO for that.

derhass
  • 43,833
  • 2
  • 57
  • 78