0

I want to reconstruct a scene from a depthmap using OpenGL. The depthmap is generated using Blender. Now I want to reproject the depthmap to 3D using the following formulas:

Z = focallength *  baseline / depth
X = u*Z / focallength
Y = v*Z / focallength

u and v are clear and I got the focal length from blenders python API. Now I am stuck with the baseline. The baseline is the length of the distance-vector between the two cameras capturing the scene. In my setup I do not have a 2nd camera. I only got the one blender camera at position (0,1,10).

glethien
  • 2,440
  • 2
  • 26
  • 36

1 Answers1

1

As far as you are dealing with camera coordinates (depth map's u,v) and the world coordinates ( 3D reconstruction's X,Y,Z) the following equation can be used:

zc[u,v]'= A [R t] [X,Y,Z]'

where zc is your depth value at u,v

A is camera calibration matrix, but obviously (in your equations), you need only information about focal length (as far as you are using ideal pinhole camera in Blender I supose).

and R,t are the rotation and translation of the world coordinate frame with respect to camera coordinate frame. If you want to have the origin of your new coordinate system in the position of the origin of the camera coordinate system you need to set your depth map camera to (0,0,0) and then you can forget R,t as well.

As you can see, you dont need any baseline value (set it to 1) in your case. Just use

zc[u,v,1]' = [f 0 0; 0 f 0; 0 0 1] [X,Y,Z,1]'

so if I am right your equations should look like:

X=zc*u/f
Y=zc*v/f
Z=zc

where zc is the depth value.

missZBH
  • 143
  • 1
  • 7
  • Nice answer. If I am using my or your way it seems that the z values reaching infinity and therefore the scene is wrong. X and Y values are working perfectly with your solution! http://www.ld-host.de/uploads/images/8c818ff0f9946ccbc653deb827b1c3fb.png – glethien Jul 08 '14 at 15:48
  • 1
    What is the format of your values in the depth map? As far as the white border of your image has correct z coordinates in the reconstruction (based on the image) and it has maxium possible value, it seems like the values in the depth map are not integers, are they? But maybe I am wrong. – missZBH Jul 09 '14 at 06:53
  • I tried both integers and floats (0-1) and integers from 0-255. Here is the depthmap I am using. https://drive.google.com/file/d/0Bx9OKnxaua8kSzdiRU5wc3ZSNVU/edit?usp=sharing – glethien Jul 09 '14 at 08:16
  • 1
    It depends on the resolution you want to acquire, you know, if you have 8bit integers you will just have 255 different z values. But the coordinates has to be scaled in the way you want them. I think it will be best to save values as the actual depth values from the scene. Is it possible? And pleas upload an image of the 3D reconstruction where the camera is placed in the correct position (the same as in the image, as far as you dont translate nor rotate the coordinate system). We will see if the x and y are correct – missZBH Jul 09 '14 at 09:17
  • x and y are correct. It is only the depth which does strange things.. https://drive.google.com/file/d/0Bx9OKnxaua8kRUxSLXlvQXBWbW8/edit?usp=sharing This shows the scene from the original camera position – glethien Jul 09 '14 at 09:51
  • 1
    Is it possible to set the camera to (0,0,0), create a depth map and then create the reconstruction? Pleas try it – missZBH Jul 09 '14 at 10:56
  • I got it now, you needed to translate your coordinate system of the camera [0,1,10] to the world coordinate system [0,0,0]. I will correct my answer. – missZBH Jul 09 '14 at 11:36