cv2 pose estimation using homography matrix

Question

I am trying to calculate the pose of image Y, given image X. Image Y is the same as image X rotated 90º degrees.

1 -So, for starters i find the matches between both images.

2 -Then i store all the good matches.

3 -The homography between the the matches from both images is calculated using cv2.RANSAC.

4 -Then for the X image, i transform the 2d matching points into 3d, adding 0 as the Z axis.

5 -Object points contain all points from matches of original image, while image points contain matches from the training image. Both array of points are filtered using the mask returned by homography.

6 -After that, i use cv2.calibrateCamera with these object points and image points.

7 -Finnaly i use cv2.projectPoints to get the projections of the axis

I know with that until step 5, the results are correct because i use cv2.drawMatches to see the matches. However this may not be the way to get what i want to achieve.

    matches = flann.knnMatch(query_image.descriptors, descriptors, k=2)

    good = []
        for m, n in matches:
            if m.distance < 0.70 * n.distance:
                good.append(m)

    current_good = good

    src_pts = np.float32([selected_image.keypoints[m.queryIdx].pt for m in current_good]).reshape(-1, 1, 2)
    dst_pts = np.float32([keypoints[m.trainIdx].pt for m in current_good]).reshape(-1, 1, 2)

    homography, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)


    test = np.zeros(((mask.ravel() > 0).sum(), 3),np.float32) #obj points
    test1 = np.zeros(((mask.ravel() > 0).sum(), 2), np.float32) #img points

    i=0
    counter=0
    for m in current_good:
          if mask.ravel()[i] == 1:
                test[counter][0] = selected_image.keypoints[m.queryIdx].pt[0]
                test[counter][1] = selected_image.keypoints[m.queryIdx].pt[1]
                test1[counter][0] = selected_image.keypoints[m.trainIdx].pt[0]
                test1[counter][1] = selected_image.keypoints[m.trainIdx].pt[1]
                counter+=1
            i+=1

    gray = cv2.cvtColor(self.train_image, cv2.COLOR_BGR2GRAY)
    gray = np.float32(gray)

    #here start my doubts about what i want to do and if it is possible to do it this way
    ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera([test], [test1],                                                                         gray.shape[::-1], None, None)

    axis = np.float32([[3, 0, 0], [0, 3, 0], [0, 0, -3]]).reshape(-1, 3)
    rvecs = np.array(rvecs, np.float32)
    tvecs = np.array(tvecs, np.float32)

    imgpts, jac = cv2.projectPoints(axis, rvecs, tvecs, mtx, dist)

However after all this, imgpts returned by cv2.projectPoints give results that don't make much sense to me, like :

[[[857.3185   109.317406]]
 [[857.2196   108.360954]]
 [[857.2846   107.579605]]]

I would like to have a normal to my image like it is shown here https://docs.opencv.org/trunk/d7/d53/tutorial_py_pose.html and i successfully got it to work using the chessboard image. But trying to adapt to a general image is giving me strange results.

cv2 pose estimation using homography matrix

0 Answers0