-1

I'm currently in the process of trying to calibrate a stereo endoscope using the openCV framework and have some trouble interpreting the results I'm getting.

I collected a clean dataset of 70 chessboard target images (size: 7x6), which are distributed approximately uniformly in the visible working space of about 30-110mm in front of the endoscope, while being randomly rotated. Clean in this context means that the corners returned by openCVs' detectChessboardCorners function are as accurate as they can get, given the image quality.

When using the standard openCV stereoCalibrate() function, without any modification to the default flags, I get reasonable results with reprojection errors mostly being < 1px.

The only issue I have with the result is that some of the parameters are not what I expected. Most notably, I know for a fact that we have a toe-in angle (y-axis rotation) of about 3.9°, whereas the openCV calibration routine always returns angles < 1.0°.

I experimented a bit and found that the only time I get the correct toe-in angle from the calibration is when I completely disable the radial distortion (setting cv2.CALIB_FIX_K1, cv2.CALIB_FIX_K2 and CALIB_FIX_K3 to 0) and only allow for tangential distortion (the p1 and p2 parameters). However, doing so results in reprojection errors of ~2px. Whatever other distortion model I use (and I think I tried all possible permutations) always results in the wrong angles (while still resulting in subpixel reprojection errors).

Has anyone a reasonable explanation for this? I'm quite confused by the fact that I get such low reprojection errors, even though the stereo camera parameters are definitely wrong.

hey0
  • 43
  • 6
  • post your data. nothing less will do. bad data is usually the culprit, even before bad code. -- low reproj error is necessary but not sufficient. if the input is a specific type of bad, reproj error can still be low. – Christoph Rackwitz Apr 03 '23 at 15:52
  • In this story, it seems that the chessboard will be very small. Is accuracy of chessboard itself OK? – fana Apr 06 '23 at 01:22

1 Answers1

0

Has anyone a reasonable explanation for this? I'm quite confused by the fact that I get such low reprojection errors, even though the stereo camera parameters are definitely wrong.

Generally speaking, depending on the object cameras are looking at, it can be difficult to distinguish between rotation and translation of the camera.
(The image change due to small rotation is very similar to that due to small translation.)

I think it's likely that such a difficult situation arises when the depth range of the data is not wide enough.
Although it is not possible depending on the Depth of Field, including the distant object in the data so that the effect of parallax can be ignored helps distinguish between rotation and translation.

fana
  • 1,370
  • 2
  • 7
  • Of course, ideally, if there is some variation in the data, it can be estimated correctly. However, in real scene, since the measurement error affects the estimation result, so the effect of the Variation have to be dominant over that of the measurement error. – fana Apr 06 '23 at 01:52