0

I am using a P3P algorithm to compute the rotation and translation of a camera given pre-mapped 3D points, and their corresponding projections on the 2D plane (PNP problem). The algorithm I am using is the one described in a CVPR 2017 paper, with implementations in both OpenCV and OpenMVG (I am using the latter). Naturally, the algorithm works inside a RANSAC framework.

Strangely, I am noticing a precise 'drift' in the position computed, that changes with the rotation angles. I.e.: If I hold the camera's position constant and just rotate it, according to the OpenCV coordinate convention, changes in pitch cause the position to drift in the Y direction, and changes in yaw cause the position to drift in the X direction (I have not tested roll yet.) When I tried to fit a curve to this dataset of pitch vs Y and yaw vs X values, I noticed a pretty constant variation. After removing scale factors and converting the angles into radians, this is the profile I see:

X ~= -4.0 * yaw_angle
Y ~= 4.0 * pitch_angle

(Translation units in meters with a scale factor of 1.0, rotation angles in radians)

Is this some sort of a commonly known transformation I am failing to account for? It's linear, so I am assuming it cannot be dependent on the rotation angle. I have skimmed through the paper and some relevant resources but I am unable to figure out the cause of this relationship.

Curve fitting results from MATLAB: Pitch curve Yaw curve

HighVoltage
  • 722
  • 7
  • 25
  • Is the output of the implementation position/orientation of the "world" in camera coordinates, or of the camera in world coordinates? – etarion Jan 23 '18 at 15:25
  • Camera in world coordinates – HighVoltage Jan 23 '18 at 15:30
  • I was asking because errors like this often are due to this switch (unless world and camera coordinate origins coincide, the world does "move" when the camera rotates from the camera's point of view). So unless you're 100% sure I'd double check :) Can be something else though, of course. – etarion Jan 23 '18 at 15:34
  • But wouldn't that be more of a rotational transformation? Even then, I am looking at the position/orientation of the camera itself and not a target: I am trying to localize a view within an existing 3D reconstruction – HighVoltage Jan 23 '18 at 15:37
  • If by "rotational transformation" you mean a trigonometric function, you're right, but sin and tan are pretty much linear from -0.3 to 0.3 radians and cos is pretty much flat so I wouldn't rule it out. If your world origin is 4 units in front of the camera and you get world-in-camera-coordinates, you'd get pretty much the effect that you're seeing. – etarion Jan 23 '18 at 15:46
  • I am going to recheck my code for that. And I forgot to add: those translations are in meters. So essentially in true world units, a rotation of about 20 degrees is causing a drift of 6 meters! – HighVoltage Jan 23 '18 at 16:04
  • You're actually using a P3P algorithm - it can be useful to state this instead of PnP to make it clear. – Toby Collins Jan 23 '18 at 16:10
  • Are you using, or have compared your results with, the author's own implementation of the algorithm in OpenCV? https://github.com/opencv/opencv/blob/638a01a014cb1972a6ffa63755474ce89e3db9d8/modules/calib3d/src/ap3p.h – Francesco Callari Jan 24 '18 at 17:09

1 Answers1

0

The problem was a bug in my code where I was accessing the translation part of the solution directly, and not the camera position, so it was indeed a missing transformation.

For future reference, the true camera position from a PNP solution needs to be computed as

C = -R' * t

R is the rotation matrix and t is the final translation vector returned by the PNP algorithm. Being a linear operator, this encodes a purely linear relationship with respect to the rotation angles.

HighVoltage
  • 722
  • 7
  • 25