2

I want to create a Depth Map in order to obtain the 3D position of each pixels so that I can have 3D position of some selected Item on my picture. In order to see if my depth data is correct, I visualize it with MeshLab.

I use the stereo data of KITTI dataset so the images are rectified and the calibration for each camera is provided.

The process is the following:

Image left + image right --> Compute disparity using Stereo Semi Global Matching (SGBM) --> Compute the depth Map using cv::reprojectImageTo3D() with the Q initialized thanks to the parameters of the calibration and thanks to this function:

 cv::stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, imgSize, R, T, R1, R2, P1, P2, Q);

My problem is the following: enter image description here enter image description here enter image description here

The road is okey but the sign board have some distortion. I don't understand and I tried to change the parameters but without success. I always have this distortion. It's annoying because I cannot compute a good 3D position of the sign board. I tried also with the classic block matching but It's the same and the results is not that good in comparison with the semi global one.

However my disparity looks like this (Which seems good to me) : enter image description here

The parameters of the disparity computation is the following:

    StereoSGBM sgbm;
    sgbm.SADWindowSize = 3;
    sgbm.numberOfDisparities = 128;
    sgbm.preFilterCap = 10;
    sgbm.minDisparity = 0;
    sgbm.uniquenessRatio = 10.0;
    sgbm.speckleWindowSize = 100;
    sgbm.speckleRange = 32;
    sgbm.disp12MaxDiff = 1;
    sgbm.fullDP = 1;
    sgbm.P1 = sgbm.SADWindowSize*sgbm.SADWindowSize*4;
    sgbm.P2 = sgbm.SADWindowSize*sgbm.SADWindowSize*32;

    sgbm(gray1, gray2, disp);

Do you have an idea why that's happened ? How I can solve that ? I would like to have a well planar surface of the sign board.

lilouch
  • 1,054
  • 4
  • 23
  • 43

1 Answers1

0

You could probably improve things slightly by playing with the SGBM parameters, but the reality is stereo data is noisy and you should not expect to get a perfectly planar sign in your point cloud.

If you are interested in finding the 3D location of the sign, segmenting the sign from the RGB image and averaging the corresponding points in the point cloud together should produce reasonable results.

Chris K
  • 1,703
  • 1
  • 14
  • 26
  • Okey, Yeah I tried to improve the parameters and It's still not good. So as you suggested, I need to take the sign board and then in the area, do an average of all the depth point, that's it ? Thanks. – lilouch Jul 14 '15 at 02:32
  • 1
    if position is the only thing you care about that should do reasonably well. In that particular image you could probably segment the red portion of the sign fairly easily by looking for all of the pixels at least .5 m above the ground plane with red channel values over a threshold and blue and green below a threshold. Of course that will not work generally. – Chris K Jul 14 '15 at 22:51
  • Thanks you for your answer. Actually, It's not only for this sign board but also for others. I'll compute the average of the depth by putting a box around the sign board. – lilouch Jul 15 '15 at 02:18