0

Let's say I have a pinhole camera with known intristic values like camera matrix and distortion coefficients. Let's say there is a point in large enough distance from the camera, so we can say it is placed in infinity.

Given image coordinates of this point in pixels, I would like to calculate camera rotation relative to the axis that connects camera and this point (so rotation is 0,0 if camera is directed at this point and it is in the optical center of the image).

How can this be done using opencv?

Many thanks!

user1431676
  • 63
  • 1
  • 8

1 Answers1

1

You need to specify an additional constraint - rotating the camera from its current pose to one that aligns the optical axis with an arbitrary ray leaves the camera free to rotate about the ray itself (i.e. it leaves the "roll" angle unspecified).

Let's assume that you want the roll to be zero, i.e. that you want the motion to be a pure pan-tilt. This has a unique solution as long as the ray you want to align to is not parallel to the vertical image axis (in which case pan and roll are the same motion).

Then the solution is computed as follows. Let's use the OpenCV camera frame: Z=[0,0,1]' (, where " ' " means transpose) be the camera focal axis, oriented going out of the lens, Y=[0,1,0]' the vertical axis going down, and X = Z x Y (where 'x' is the cross product) the horizontal camera axis going toward the right of the image. So "pan" is a rotation about Y, "tilt" is a rotation about X.

Let U = [u1, u2, u3]', with || u || = 1 be the ray you want to rotate to. You want to apply a pan that brings Z onto the plane Puy defined by the vectors u and Y, then apply a tilt that brings Z onto u.

The angle of the first rotation is (angle between Z and Puy) = [90 deg - (angle between Z and Y x U)]. this is because Y x U is orthogonal to Puy. Look up the expressions for computing the angle between vectors on Wikipedia or elsewhere online. Once you have the angle (or its cosine and sine), the rotation about Y can be expressed as a standard rotation matrix Ry.

The angle of the second rotation, about X after once Z is onto Puy, is the angle between vector Z and U after Ry is applied to Z, or equivalently, between Z and inv(Ry) * U. Compute the angle between the vector, and use to build a standard rotation matrix about X, Rx

The final transformation is then Rx * Ry.

Francesco Callari
  • 11,300
  • 2
  • 25
  • 40
  • Thanks, you are right, I didn't mentioned the roll*0. The problem is, I don't know the 3d coordinates, only 2d pixel coordinates. This, together with camera matrix, is my starting point. – user1431676 Jun 19 '14 at 12:23
  • I would basically need a inverse function from projectPoints(). Since I didn't find it in opencv, I'm tryng to solve this by searching through angles, until I find the pair that projects to given pixel. I will post a solution if it will be successful. – user1431676 Jun 19 '14 at 12:29
  • You do not need the third coordinate, just the pixel is enough. Let U = [px, py, 1], then normalize to unit length – Francesco Callari Jun 20 '14 at 04:42