2

I have a camera mounted on a post which is looking at a specific field. I know the exact real world position of the camera C and I know the exact size and real world position of the field P1, P2, P3, P4, W, H.

The problem is that the camera may rotate and I want to get exact rotation values (yaw, pitch, roll).

I can easily find all the corners and I used cv2.getPerspectiveTransform to get the perspective matrix H but I don't know how to extract rotation values/matrix from this.

I have a solution which gets some approximated values but it is very "heuristic" and seems way too complicated. I could also use cv2.solvePnP but it "tries" to estimate the position which is known so it also looks not optimal.

My common sense tells me that I should just decompose matrix H but I am just not sure how. I mean it is a 3x3 matrix and I should use 4x4. I could add some ones or zeros but I would really like to understand what I am doing instead of "just to have a working solution".

I would be really glad if someone could help me with this and give a simple explanation. I attach 3d projections for better understanding.

EDIT: I also did the calibration and have intrinsic params/matrix but for now I want to solve it on a "virtual" camera.

Thanks in advance!

enter image description here enter image description here enter image description here

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Dev_IL
  • 252
  • 1
  • 15
  • solvePnp should work perfectly, if your image and object points (object size) are known, as well as camera intrinsics. But keep in mind that solvePnp computes the object pose, not the camera pose, but you can get the one from the other. – Micka Nov 27 '22 at 15:03
  • @Micka You are right - this will work. However there is one additional piece of information I have - 3D world position of the camera - which I will NOT use in this case. I could just put those 4 mapping points and get the result. But If I get it correctly If I have position of the camera then I would only need 3 mapping points. But I have no idea how to properly construct matrix equations for this.. I – Dev_IL Dec 03 '22 at 17:15
  • 1
    You could precompute the ground plane to image perspective transformation, then with 3 rectangle points you can compute the rigid transformation in 2D ground plane space, by 1. compute image to ground coordinates and then rigid transform. – Micka Dec 03 '22 at 17:30
  • 1
    Basically you would with the intrinsics and extrinsics define 3D rays from camera center through a pixel in the image plane and follow it through 3D space until it hits the ground plane (z == 0) then you have a pixel-ground-plane-correspondence. With 4 of them you have the perspective homography. – Micka Dec 04 '22 at 09:07
  • @Micka thx for these hints! Is there a chance that you could describe it a bit more and/or add some equations/examples? I kinda know what you mean but since I have a very poor 3d "spatial recognition" it's hard for me to really understand and use it in my case :/ I get it that since I know 3d location of 3 object's points and I also know 3d location of the camera I could construct a tetrahedron and somehow calculate it's rigid transform from "ground plane" but I don't know how to calculate it and hot to incorporate those 2d image "mapped" points with this. – Dev_IL Dec 07 '22 at 19:41
  • EDIT: Would estimateAffine3D be helpful to get this rigid transformation? I have 4 3d points (camera points + 3 object points) but how to get "camera coordinates" points? Would it be (ui, vi, zi) ? Where ui,vi are image coords and zi = 3d_dist(C, Xi) – Dev_IL Dec 07 '22 at 20:48

1 Answers1

1

Ok, I managed to find the solution - I used Grunert's Method as a reference.

So TLDR: Since I have the translation vector and 3 corresponding 3D-2D points so I can skip the first part of p3p problem (finding distances from camera to 3 points) and just focus on the second part which is rigid body transformation and getting R matrix from this equation:

enter image description here

It's easy to do using SVD decomposition and I used this as a reference.

@Micka thx again for your hints!

Dev_IL
  • 252
  • 1
  • 15