4

Background:

For simplicity, I have 3 Cameras. Each one of them is calibrated by its own (its interstice parameters are known and accurate enough)

They all share almost the same view. I want to use this system of 3 Cameras for 3D reconstruction. I have calibrated each pair alone (1 & 2, 2 & 3 and 1 & 3). Thus resulted in 3 transformation matrices. Theoretically, these 3 transformation matrices should has the following propriety:

T13 = T12 * T23

Or in other formula:

 T31 * T12 * T23 = Identity

However, practically, this is not true. The result is a bit shifted from the identical matrix due to calibration error.

Error = [T31 * T12 * T23] - [Identity]        Eq.3

Problem:

I feel calibrating each pair of cameras alone is such a bad idea. By applying a some kind of global optimization that fuses the whole three pairs together trying to minimizing the error of the equation (3) should give better results.

My question is:

Is there any known previous work on this problem. Any Idea how to use OpenCV to achieve my goal so I do not reinvent the wheel again.

Humam Helfawi
  • 19,566
  • 15
  • 85
  • 160
  • maybe you can use some kind "bundle adjustement" which is used in image mosaicing more or less directly. – Micka Nov 13 '17 at 19:15
  • It would help if you explained what further details you need. Your problem needs more than a few lines of code pasted in a SO answer. AFAICS it cannot be done entirely in OpenCV, since OpenCV has no bundle adjustment API, and it doesn't even expose the LM optimizer used by its camera calibration routines. – Francesco Callari Nov 13 '17 at 21:10
  • @FrancescoCallari Thanks. Yes I was looking for sth like the LM optimizer in OpenCV. The idea came to my mind from slamming actually. I was looking for solution like this in OpenCV. It seems I have to build it my self or I should use 3D calibration object and get some help from PCL (They have LUM support already). – Humam Helfawi Nov 13 '17 at 22:27
  • For a small number of cameras any generic LM implementation will do (see Numerical Recipes, or minpack, or even scipy.optimize.least_squares). Much better to use a real bundler, e.g. Ceres, which can nicely handle the parametrization of camera rotations so you won't have to reinvent the wheel in your cost function. – Francesco Callari Nov 13 '17 at 22:52

1 Answers1

5

Well, of course sets of many cameras have been calibrated jointly in the past (did you watch the "bullet shot" effect in The Matrix ? This is how it was done by the team at Manex first: https://www.youtube.com/watch?v=uPNBdDNZbYk )

You can use your initial pairwise calibration as the initialization point for a round of bundle adjustment. It is preferable to use a rigid 3D object (not a planar "checkerboard" target, you need a target with actual depth) visible in several cameras in all of the images, and use many images spanning the field of view and the depths of interest. If it's a one-off project, the image matching and bundling can easily be done in a semiautomatic tool with a good GUI. I have satisfactorily used Blender (www.blender.org) for these tasks in the past. Look online for tutorials on using Blender for matchmoving.

Francesco Callari
  • 11,300
  • 2
  • 25
  • 40