1

Given a couple of stereo images from a calibrated camera pair one usually wants to apply rectification. As stated in "Computing Rectifying Homographies for Stereo Vision" by Loop and Zhang (link):

In general, epipolar lines are not aligned with coordinate axis and are not parallel. Such searches are time consuming since we must compare pixels on skew lines in image space. These types of algorithms can be simplified and made more efficient if epipolar lines are axis aligned and parallel. This can be realized by applying 2D projective transforms, or homographies, to each image. This process is known as image rectification. The pixels corresponding to point features from a rectified image pair will lie on the same horizontal scan-line and differ only in horizontal displacement.

To do 3D reconstruction one has to run a matching algorithm on the images, to solve the famous correspondence problem.

Are the matching algorithms applied to the original images or to the rectified ones? In fact the distortion introduced by rectification may cause pattern recognition to fail, right?

Please share any reference if available. Thanks.

decadenza
  • 2,380
  • 3
  • 18
  • 31

1 Answers1

2

Most matching algorithms (like "semi global block matching", used in opencv) require rectified stereo image pairs in order to function properly. So the matching algorithms are applied to the rectified images.

What the rectification actually does, is to warp the image, such that epipolar lines run horizontally in both images. That means, that objects in both images are in one vertical image plain after rectification. This way matching algorithms only have to check for korrespondence along image lines instead of the whole image.

The "distortion" introduced by rectification is usually only composed of geometric and affine image transformations. The images are transformed, such that the are better aligned and as such the pattern recognition of matching algorithms perfoms better after rectification.

A possible source for more Information could be "Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003."

M. Martin
  • 673
  • 1
  • 13
  • 23