1

This is a pretty straightforward question (I hope). The following is from 3D reconstruction from Multiple Images, Moons et al (Fig 2-13, p. 348):

Projective 3D reconstruction from two uncalibrated images

Given: A set of point correspondences m1 in I1 and m2 in I2 between two uncalibrated images I1 and I2 of a static scene.

Aim: A projective 3D reconstruction ^M of the scene.

Algorithm:

  1. Compute an estimate ^F for the fundamental matrix
  2. Compute the epipole e2 from ^F
  3. Compute the 3x3-matrix
    ^A = −(1/||e2||2) [e2]x ^F
  4. For each pair of corresponding image points m1 and m2, solve the following system of linear equations for ^M :
    ^p1 m1 = ^M and ^p2 m2 = ^A ^M + e2
    ( ^p1 and ^p2 are non-zero scalars )

[I apologize for the formatting. I don't know how to put hats over characters.]

I'm pretty much OK up until step 4. But it's been 30+ years since my last linear algebra class, and even then I'm not sure I knew how to solve something like this. Any help or references would be greatly appreciated.

By the way, this is sort of a follow-on to another post of mine:

Detecting/correcting Photo Warping via Point Correspondences

This is just another way to try to solve the problem.

Community
  • 1
  • 1
Jabberwock
  • 173
  • 1
  • 9

1 Answers1

1

Given a pair of matching image points m1 and m2, the two corresponding rays from the optical centers are unlikely to intersect perfectly due to noise in the measurements. Consequently a solution to the provided system should instead be found in the (linear) least square sense i.e. find x = argmin_x | C x - d |^2 with (for instance):

      /           0 \ /    \
      |  I  -m1   0 | |  M |
C x = |           0 | |    |
      |       0     | | p1 |
      |  A    0 -m2 | \ p2 /
      \       0     /

and

    /  0  \
    |  0  |
d = |  0  |
    |     |
    | -e2 |
    \     /

The problem has 5 unknowns for 6 equations.

A possible alternative formulation exploits the fact that m1 and m2 are collinear with M so m1 x M = 0 and m2 x (A M + e2) = 0 yielding the linear least squares problem x = argmin_x | C x - d |^2 with:

    / [m1]x   \ /   \
C = |         | | M |
    \ [m2]x A / \   /

and

    /     0    \
d = |          |
    \ -m2 x e2 /

where [v]x is the 3 x 3 matrix of the cross product with v. The problem has 3 unknowns for 6 equations which can be reduced to 4 only by keeping non-linearly dependent ones.

user3146587
  • 4,250
  • 1
  • 16
  • 25
  • Thanks! What I had blundered through was very similar to your first solution. Unfortunately, the sample points I tried with my solution didn't work out. I'll have to compare it more closely with yours. Your second solution looks interesting also. – Jabberwock Jan 21 '17 at 03:49
  • Sorry if I'm being dense, but I'm still not sure how to proceed. I see how you got `C`,`x` & `d`. I know how to solve something like `Cx=d` if `C` is square and invertible., but I'm lost on something like `argmin_x|Cx - d|^2`. A little but of Matlab code or a link to a good reference would certainly help. – Jabberwock Jan 22 '17 at 19:24
  • A solver will usually rely on a QR or an SVD decomposition of the matrix C (see the [Wikipedia article on linear least squares](https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)) for details). NumPy provides the `linalg.lstsq` function. I am not familiar with MATLAB, but it seems `x = C\d` should work. – user3146587 Jan 22 '17 at 20:07
  • Thanks. I was just about to reply that Wikipedia has a fairly understandable article on linear least squares. And thanks for the NumPy hint - that's what I really needed. – Jabberwock Jan 23 '17 at 02:22