3D Reconstruction from multiple calibrated views

Question

I have a calibrated camera whose intrinsics were calculated prior to doing an initial two view reconstruction. Suppose I have 20 images around a static, rigid body all taken with the same camera. Using the first two views and a ground-truth measurement of the scene, I have the

1) initial reconstruction using Stewenius 5 point algorithm to find E (essential matrix).

2) camera matrices P1 and P2 where the origin is set to that of camera P1.

My question is, how would I add more views? For the first two views, I found the feature points by hand since I found that MATLAB feature-detectors and matchers were outputting false correspondences.

Do I continuously do two-view reconstructions to get the other camera extrinsics i.e. P1 and P3, P1 and P4...P1 and P20; all using the same feature points as that of P1-P2? Wouldn't there be some sort of error propagation with this approach? The reason for using P1 as a reference is because it is chosen to be at the world origin.

I do have a procedure to bundle adjust after I acquire all initial estimates for the camera extrinics, but my problem is getting the initial camera matrices P3...P20.

Thanks in advance!

Answer 1

You start by obtaining pairwise calibration P1-P2, P2-P3, P3-P4 ... using feature points of the corresponding pair. You need using some sort of RANSAC to get rid of false correspondences here or do matching manually between all pairs. The you need to put all cameras to the common coordinate frame. Say we select P1 as a key camera. To add third camera P3 to the pair P1-P2 you need to calculate rotation delta between P2 and P3 from pairwise calibration, Delta2-3 and then apply it to known camera matrix of P2. And so on until all camera matrices in the common coordinate frame. The you do bundle adjustment.