0

I have 4 different views of an object, each with its corresponding K and M camera matrices (intrinsic and extrinsic parameters).

I have 6 3D points which i marked on each of the 2D views (so 24 2d points).

What i am trying to do to back-project those 2D points to 3D and get my 6 3D points in space.

Most papers show how to do it when you have 2 views and their matrices. However, I have 4 views. I am using OpenCV in Python.

As a first approach maybe i could triangulate using all combinations of 2 views, so i get 4choose2 = 6 sets of 3D points and then I average over all triangulations.

Is there a better approach that would be more advisable? Do you think i should use something other than OpenCV? (it still has to be Python though)

Roulbacha
  • 457
  • 7
  • 20
  • OpenCV's `cvTriangulatePoints()` supports only **two-view** triangulation. Major bummer **:|** I've added a [feature request](http://github.com/opencv/opencv/issues/13077) – Nirmal Nov 08 '18 at 10:20

1 Answers1

1

I see two solutions for your problem.

1. Analytic Solution

If you do the math, you find that each 2D to 3D correspondence adds 2 equations to the triangulation system. With 6 views you will then get a system with 12 equations and 3 unknowns. This should be the most efficient implementation, but could be unstable when noisy inputs are expected.

2. Triangulate + Optimize

Compute an estimated 3D point with classic 2-View triangulation. Use non-linear optimization to compute the 3D point that minimized the reprojection error to all 6 views. This is my recommend solution, because it should be stable and everything is already implemented in OpenCV.

dari
  • 2,255
  • 14
  • 21
  • For the second method, would you be referring to this function by any chance? https://docs.opencv.org/3.0-beta/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#correctmatches – Roulbacha Mar 07 '18 at 14:44
  • No, you can use that, but it is not enough. You need an optimization over multiple cameras and points. The keyword here is Bundle Adjustment. – dari Mar 09 '18 at 17:57
  • I am given the projection matrices and corresponding points, i think this is more appropriate https://docs.opencv.org/3.3.0/d0/dbd/group__triangulation.html#ga211c855276b3084f3bbd8b2d9161dc74 – Roulbacha Mar 09 '18 at 19:18
  • That method is the first solution of my answer. You can try that, should be a good starting point. – dari Mar 09 '18 at 21:21