Comparing Kinect depth to OpenGL depth efficiently

Question

Background:

This problem is related with 3D tracking of object.

My system projects object/samples from known parameters (X, Y, Z) to OpenGL and try to match with image and depth informations obtained from Kinect sensor to infer the object's 3D position.

Problem:

Kinect depth->process-> value in millimeters

OpenGL->depth buffer-> value between 0-1 (which is nonlinearly mapped between near and far)

Though I could recover Z value from OpenGL using method mentioned on http://www.songho.ca/opengl/gl_projectionmatrix.html but this will yield very slow performance.

I am sure this is the common problem, so I hope there must be some cleaver solution exist.

Question:

Efficient way to recover eye Z coordinate from OpenGL?

Or is there any other way around to solve above problem?

Take a look at OpenCV. I have yet to use it with OpenGL, but OpenCV can do most operations like this in almost realtime. — Blender, Nov 25 '11 at 17:26
@Blender: Can you point me the docs on that ... I also searched there but with no luck. Thanks! — Rudi, Nov 25 '11 at 17:29
Hi @Rudi, could you please give an update with your solution? Thanks! — dim_tz, Jul 24 '13 at 18:36

score 2 · Accepted Answer · answered Nov 25 '11 at 17:57

Now my problem is Kinect depth is in mm

No, it is not. Kinect reports it's depth as a value in a 11 bit range of arbitrary units. Only after some calibration has been applied, the depth value can be interpreted as a physical unit. You're right insofar, that OpenGL perspective projection depth values are nonlinear.

So if I understand you correctly, you want to emulatea Kinect by retrieving the content of the depth buffer, right? Then the most easy solution was using a combination of vertex and fragment shader, in which the vertex shader passes the linear depth as an additional varying to the fragment shader, and the fragment shader then overwrites the fragment's depth value with the passed value. (You could also use an additional render target for this).

Another method was using a 1D texture, projected into the depth range of the scene, where the texture values encode the depth value. Then the desired value would be in the color buffer.

Comparing Kinect depth to OpenGL depth efficiently

1 Answers1