5

I want to estimate the orientation angles of a moving camera. I have the camera intrinsics K and can track keypoints to calculate the essential matrix E and from there on also the rotational matrix R with respect to the previous frame of the video.

E, _ = cv2.findEssentialMat(kpn_cur, kpn_ref, focal=1, pp=(0., 0.), method=cv2.RANSAC, prob=kRansacProb, threshold=kRansacThresholdNormalized)
_, R, t, mask = cv2.recoverPose(E, kpn_cur, kpn_ref, focal=1, pp=(0., 0.))

(using already normalized keypoints kpn_cur and kpn_ref, calculated with the camera intrinsics K)

This enables me to get the direction and location of the frames with respect to each other, but what I really want is the position (and orientation) of the camera with respect to ground plane of each frame.

I know there are ways using vanishing points to calculate the camera orientation (also by calculating the rotation matrix from vanishing points).

Whats the way to calculate the orientation with respect to ground plane from the given rotation matrix?

Thanks!

Alexander Riedel
  • 1,329
  • 1
  • 7
  • 14
  • Without encoder inside rotator; it seems impossible. Image processing based assumption will not work properly also many crushing possiblity exists – Yunus Temurlenk Feb 18 '21 at 05:58
  • I also think that this is not possible. For my understanding you would need 3D coordinates from the ground plane which are matched with the corresponding 2D image points. – Grillteller Feb 23 '21 at 12:30
  • 2
    If you can stick and Aruco tag on the ground and catch it in a frame, you can use that to calculate your orientation relative to the ground. From there on, since you have the rotation and translation from one frame to the next, you can "dead reckon" your camera orientation for a while. You'll accumulate error while doing this until you spot an Aruco tag again. – Ian Chu Feb 24 '21 at 05:44
  • 2
    what exactly is your main problem? The change of camera pose or the change of the ground plane? If it is just the change of camera pose, and you know your initial camera pose, you can track the current pose by incrementally updating it with R and T from recover pose (just multiplication of 4x4 matrices in the right order - see virtual camera models like openGL). But you will get some drift and errors summing up and probably your T and R can be very wrong for some of the images. – Micka Feb 25 '21 at 08:35

0 Answers0