Calculating the essential matrix from two sets of corresponding points

Question

I'm trying to reconstruct a 3d image from two calibrated cameras. One of the steps involved is to calculate the 3x3 essential matrix E, from two sets of corresponding (homogeneous) points (more than the 8 required) P_a_orig and P_b_orig and the two camera's 3x3 internal calibration matrices K_a and K_b.

We start off by normalizing our points with

P_a = inv(K_a) * p_a_orig

and

P_b = inv(K_b) * p_b_orig

We also know the constraint

P_b' * E * P_a = 0

I'm following it this far, but how do you actually solve that last problem, e.g. finding the nine values of the E matrix? I've read several different lecture notes on this subject, but they all leave out that crucial last step. Likely because it is supposedly trivial math, but I can't remember when I last did this and I haven't been able to find a solution yet.

Answer 1

This equation is actually pretty common in geometry algorithms, essentially, you are trying to calculate the matrix X from the equation AXB=0. To solve this, you vectorise the equation, which means,

vec() means vectorised form of a matrix, i.e., simply stack the coloumns of the matrix one over the another to produce a single coloumn vector. If you don't know the meaning of the scary looking symbol, its called Kronecker product and you can read it from here, its easy, trust me :-)

Now, say I call the matrix obtained by Kronecker product of B^T and A as C. Then, vec(X) is the null vector of the matrix C and the way to obtain that is by doing the SVD decomposition of C^TC (C transpose multiplied by C) and take the the last coloumn of the matrix V. This last coloumn is nothing but your vec(X). Reshape X to 3 by 3 matrix. This is you Essential matrix.

In case you find this maths too daunting to code, simply use the following code by Y.Ma et.al:

%  p are homogenius coordinates of the first image of size 3 by n
%  q are homogenius coordinates of the second image of size 3 by n

function [E]  = essentialDiscrete(p,q)

n = size(p);
NPOINTS = n(2);

% set up matrix A such that A*[v1,v2,v3,s1,s2,s3,s4,s5,s6]' = 0
A = zeros(NPOINTS, 9);

if NPOINTS < 9
     error('Too few mesurements')
     return;
end

for i = 1:NPOINTS
  A(i,:) = kron(p(:,i),q(:,i))';
end
r = rank(A);

if r < 8 
  warning('Measurement matrix rank defficient')
  T0 = 0; R = [];
end;

[U,S,V] = svd(A);

% pick the eigenvector corresponding to the smallest eigenvalue
e = V(:,9);
e = (round(1.0e+10*e))*(1.0e-10);
% essential matrix 
E = reshape(e, 3, 3);

Answer 2

You can do several things:

• The Essential matrix can be estimated using the 8-point algorithm, which you can implement yourself.
• You can use the estimateFundamentalMatrix function from the Computer Vision System Toolbox, and then get the Essential matrix from the Fundamental matrix.
• Alternatively, you can calibrate your stereo camera system using the estimateCameraParameters function in the Computer Vision System Toolbox, which will compute the Essential matrix for you.