1

In OpenCV APIs, there are cv::stereoCalibrate() and cv::fisheye::stereoCalibrate() for calibrating normal stereo cameras and fisheye stereo cameras repectively. For both APIs, the stereo camera pair use the same camera model. In other words, both stereo cameras must use normal camera model (6 radial distortion coefficients + 2 tangential ones) for cv::stereoCalibrate() or fisheye camera model (4 fisheye distortion coefficients) for cv::fisheye::stereoCalibrate().

Is there any way to calibrate a stereo camera pair where one use normal camera model and the other use fisheye camera model via OpenCV?

herohuyongtao
  • 49,413
  • 29
  • 133
  • 174
  • You could perform another camera calibration on the undistorted fisheye image and feed the undistorted camera to your stereo calibration. I guess this could work but has the addutional overhead of a remap call for every captures fisheye image and might give disadvantages because of altered resolution. – Micka Dec 10 '22 at 10:16
  • 1
    @Micka Thanks for your suggestion. This might work. I'm also trying undistorting points first instead of undistorting the images before the stereo calib step, which somehow bypasses the resolution change issue. – herohuyongtao Dec 10 '22 at 10:26

1 Answers1

0

For anyone find this question helpful, OpenCV doesn't natively support stereo calibration (i.e., via stereoCalibrate() API) for mixed camera models (e.g., one is normal, and the other fisheye). However, this can be done by the following steps:

  1. Calibrate each camera's intrinsics via API cv::calibrateCamera() or cv::fisheye::calibrate().
  2. Undistort all image points for both cameras use previous intrinsics via API cv::undistortPoints() or cv::fisheye::undistortPoints().
  3. Calibrate the extrinsics for the stereo pair using the image points after undistortion in previous step and "perfect intrinsics" for both cameras via API cv::stereoCalibrate() or cv::fisheye::stereoCalibrate() (with CALIB_FIX_INTRINSIC flag to compute extrinsics only). A camera with perfect intrinsics means its camera matrix is [1 0 0; 0 1 0; 0 0 1] and has zero distortion (distortion coefficients are all 0).

    Note: It doesn't matter to use cv::stereoCalibrate() or cv::fisheye::stereoCalibrate() in this step, which will result in the same extrinsics.

herohuyongtao
  • 49,413
  • 29
  • 133
  • 174