6

I'm having trouble undistorting points on an image taken with a calibrated camera using the Python bindings for OpenCV. The undistorted points have entirely different coordinates than the original points detected in the image.

Here's the offending call:

undistorted = cv2.undistortPoints(image_points,
                                  camera_matrix,
                                  distortion_coefficients)

where image_points is a numpy array of detected chessboard corners returned by cv2.findChessboardCorners and reshaped to match the dimensional requirements of cv2.undistortPoints, and camera_matrix and distortion_coefficients were returned by cv2.calibrateCamera.

camera_matrix and distortion_coefficients seem to me to be okay, and so do image_points. Nevertheless, distorted seems to have no relationship to image_points. Here's a summary of the values:

>>> image_points
array([[[ 186.95303345,  163.25502014]],

       [[ 209.54478455,  164.62690735]],

       [[ 232.26443481,  166.10734558]],

       ..., 

       [[ 339.03695679,  385.97784424]],

       [[ 339.20108032,  400.38635254]],

       [[ 339.13067627,  415.30780029]]], dtype=float32)
>>> undistorted
array([[[-0.19536583, -0.07900728]],

       [[-0.16608481, -0.0772614 ]],

       [[-0.13660771, -0.07537176]],

       ..., 

       [[ 0.00228534,  0.21044853]],

       [[ 0.00249786,  0.22910291]],

       [[ 0.00240568,  0.24841554]]], dtype=float32)
>>> camera_matrix
array([[ 767.56947802,    0.        ,  337.27849576],
   [   0.        ,  767.56947802,  224.04766824],
   [   0.        ,    0.        ,    1.        ]])
>>> distortion_coefficients
array([[ 0.06993424, -0.32645465,  0.        ,  0.        , -0.04310827]])

I'm working with reference C code and everything matches up until I make that call. What's going wrong?

Daniel Lee
  • 2,030
  • 1
  • 23
  • 29

1 Answers1

13

I think you forgot to specify the new camera matrix in your call to undistortPoints. If you look at the documentation of the function, it says that the signature is:

Python: cv.UndistortPoints(src, dst, cameraMatrix, distCoeffs, R=None, P=None) → None

where dst is the array of points after undistortion and "if P is identity or omitted, then it contains normalized point coordinates", meaning before projection in the image using the calibration matrix.

The function should do what you expect if you set P to your cameraMatrix.

BConic
  • 8,750
  • 2
  • 29
  • 55
  • 4
    That did it! I used the (undocumented) signature from ``cv2``: ``cv2.undistortPoints(image_points, camera_matrix, distortion_coefficients, P=camera_matrix)`` – Daniel Lee Feb 26 '14 at 19:19
  • I am a little confused about what undistortPoints is doing exactly, what does the documentation mean by ideal image coordinates? – r4bb1t Apr 24 '20 at 17:58
  • 1
    @PyWalker2797 ideal image coordinates are the 2D image coordinate which would have been observed if the lens was ideal with zero distortion. For a real non-ideal lens, you have to account for distortion, which adds an offset on the image coordinates. More details on the camera projection model can be found [here](https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html). – BConic Apr 27 '20 at 16:08
  • So the undistortPoints gives us back the image coordinates according to zero-lens distortion? – r4bb1t Apr 27 '20 at 23:13