I'm new to stereo calibration and tried to get the matrices of a stereo camera and then remap the images, so I could use them for depth-map generation.
I used the sample images left*.jpg
and right*.jpg
and parts of this code: http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_calib3d/py_calibration/py_calibration.html#calibration (and also from samples/python/calibrate.py)
So my code is the following:
import cv2
...
img_names = [glob('../data/left*.jpg'), glob('../data/right*.jpg')]
obj_points = [[], []]
img_points = [[], []]
h = ... #
w = ... #
... # get the obj_points and img_points from both image sets like in the examples
# This works fine, I viewed the results with cv2.drawChessboardCorners
rms_l, camera_matrix_l, dist_coeffs_l, _, _ = cv2.calibrateCamera(obj_points[0], img_points[0], (w, h), None, None)
rms_r, camera_matrix_r, dist_coeffs_r, _, _ = cv2.calibrateCamera(obj_points[1], img_points[1], (w, h), None, None)
stereo_flags = 0
stereo_flags |= cv2.CALIB_USE_INTRINSIC_GUESS
stereo_flags |= cv2.CALIB_FIX_INTRINSIC
# run stereo calibration
rms_stereo, camera_matrix_l, dist_coeffs_l, camera_matrix_r, dist_coeffs_r, R, T, E, F = cv2.stereoCalibrate(obj_points[0], img_points[0], img_points[1], camera_matrix_l, dist_coeffs_l, camera_matrix_r, dist_coeffs_r, (w, h), flags=stereo_flags)
# run stereo rectification
rectification_matrix_l, rectification_matrix_r, projection_matrix_l, projection_matrix_r, _, _, _ = cv2.stereoRectify(camera_matrix_l, dist_coeffs_l, camera_matrix_r, dist_coeffs_r, (w, h), R, T)
map1, map2 = [], []
map1[0], map2[0] = cv2.initUndistortRectifyMap(camera_matrix_l, dist_coeffs_l, rectification_matrix_l, projection_matrix_l, (w, h), cv2.CV_16SC2)
map1[1], map2[1] = cv2.initUndistortRectifyMap(camera_matrix_r, dist_coeffs_r, rectification_matrix_r, projection_matrix_r, (w, h), cv2.CV_16SC2)
print ... # see below
for in in range(2):
img = cv2.imread(img_names[i][0], 0)
imgt = cv2.remap(img, map1[i], map2[i], cv2.INTER_LINEAR)
cv2.imshow('image', imgt)
cv2.waitKey(0)
But then, my the resulting images are very strange (left and right):
Now is my question, what can be my mistake? Did I calculated the matrices not correct or inserted wrong parameters? I saw many examples which are using those functions in a similar way, but altough their results are not like mine.
EDIT: Here are my matrices:
---------- Camera Stereo ----------
RMS: 54.0798238093
camera matrix left:
[[ 532.80990768 0. 342.49522241]
[ 0. 532.93344826 233.88792572]
[ 0. 0. 1. ]]
distortion coefficients left: [ -2.81325825e-01 2.91151900e-02 1.21234424e-03 -1.40823847e-04
1.54861062e-01]
camera matrix right:
[[ 537.43775429 0. 327.6113408 ]
[ 0. 536.95118843 248.89561922]
[ 0. 0. 1. ]]
distortion coefficients right: [-0.29741745 0.14930176 -0.00077008 0.00032599 -0.06565751]
R:
[[ 0.81520177 0.24703546 0.52385071]
[ 0.42427129 0.36098612 -0.83047149]
[-0.39425873 0.89925664 0.18946647]]
Rectification matrix Left :
[[ 0.84817656 0.5225405 -0.08687895]
[-0.44783099 0.61975759 -0.64447493]
[-0.28292036 0.58553561 0.75967369]]
Rectification matrix Right :
[[ 0.77500946 0.62063732 0.11903635]
[-0.54957916 0.56894011 0.61177602]
[ 0.31196648 -0.5395521 0.7820233 ]]
Projection Matrix Left :
[[ 448.32924881 0. 328.40793514 0. ]
[ 0. 448.32924881 215.26811713 0. ]
[ 0. 0. 1. 0. ]]
Projection Matrix Right :
[[ 4.48329249e+02 0.00000000e+00 3.28407935e+02 0.00000000e+00]
[ 0.00000000e+00 4.48329249e+02 2.15268117e+02 1.97345858e+04]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]