6

I am looking for a method or an algorithm which would allow me to recognize and pair sets of points from two different images, for the purpose of stereo vision. picture

The attached picture presents what I have at the moment: 2 cameras are aligned on the Y axis and slightly offset on the X axis, looking at a set of points. I am able to track and get the 2D positions of each of these points on the two camera images (IMG0 and IMG1).

As such, I have two lists of 2D coordinates:

L0 = { a0, a1, a2, a3, a4, a5, a6 }
L1 = { b0, b1, b2, b3, b4, b5, b6 }

Now, in order to carry out triangulation to get the 3D position of each point, I need to know which point on image IMG1 corresponds to a which point on IMG0. Both camera see the exact same set of point, with the same overall shape, but obviously, due to slight distortion and to the camera being offset on the horizontal, the 2D coordinates do not match from an image to the other.

Ideally, the point matching algorithm I am looking for would result in a list such as:

List = {a0-b0, a1-b1, a2-b2,...}

The order of the list doesn't matter as long as I am sure each point is paired with the right one from the second image.

I have been looking at several papers presenting stereo mapping algorithms, but I didn't find anything relevant to my problem as most algorithm are based on heavy image feature recognition, which is not suitable in my case where I want to quickly process everything in real-time. The closest solution I seemed to find is the point matching algorithm presented here, but once again this seems too heavy for my issue.

Any help would be greatly appreciated.

Code Lღver
  • 15,573
  • 16
  • 56
  • 75
user2187623
  • 63
  • 1
  • 3
  • when I searched online I was able to get many relevant links (mostly implemented in matlab). Eg: http://www.mathworks.com/matlabcentral/fileexchange/28522-stereo-matching . – ElKamina Mar 19 '13 at 18:49
  • How close can the points be to the camera? If the points are very close to the camera, or if the cameras are very far apart, the shape of the point clouds could be drastically different, and could require a different matching algorithm. – mbeckish Mar 19 '13 at 19:51

1 Answers1

5

First of all you should make sure you understand the basic concepts of epipolar geometry, especially the concept of an epipolar line.

In a nutshell: Say you have a 3D point P that projects to a 2D point q in the image of a camera A. Now you have a second camera, call it B, and you want to find the image of P in B. Epipolar geometry tells you that the possible locations of the image of P in B given q are limited to a line, called the epipolar line. It also tells you that (and how) you can compute this line from q and the calibration of your cameras, using what is called the Fundamental matrix.

For your problem this has the following implications:

Let q be a point from list L0.

  • If there is a single point in your list L1 which is on the epipolar line of q in the second image, then this is the correct correspondence for q.
  • If there are several points on the epipolar line, then your problem cannot be solved with the given information. In this case you need to make heuristic assumptions about the distribution of your points in 3D, since you get one possible 3D point for each pairing of q with a point on the epipolar line.

If your cameras are only offset along the X axis, and if they are identically oriented (i.e. the image planes are parallel), they it is your lucky day: In this special camera configuration, the epipolar lines are horizontal, i.e. for a point (x,y) the epipolar line is the line of all points with arbitrary X coordinate and Y coordinate y.

Note that in practice you also face the problem that points are unlikely to be exactly on epipolar lines due to measurement errors and numerical issues.

DCS
  • 3,354
  • 1
  • 24
  • 40
  • Thank you very much. Yes, my cameras are only offset along the X axis. So presumably, for a given point in L0, I guess I can check L1 for all the points that have similar Y-position (I say similar because it won't be exactly the same, as you said, due to potential measurements errors). In the case I have more point matching on the line, I can then sort them using the X position. I will give it a go, but I think this will work. – user2187623 Mar 20 '13 at 09:14
  • This should work, but, as I said: With more than one candiate on the epipolar line, you need to think which point is more plausible given the specific scene you are reconstructing. I would triangulate all candidates and then look at the 3D points to chose. – DCS Mar 20 '13 at 09:48
  • If you have more than one point in L1 that lies on the same epipolar line generated by a point in L0, I would suggest creating a graph in which there is a vertex for each point in L0, a vertex for each point in L1, and an edge between two vertices whenever one lies on the other's epipolar line. You can then find a best-possible matching of points by running an algorithm for bipartite maximum matching on this graph. – j_random_hacker Mar 20 '13 at 11:38
  • @j_random_hacker: I think he has as many point in L0 as in L1, so you would have to introduce weights in order to favor one matching over another. Does bipartite maximum matching support weights, or is it strictly combinatorial? – DCS Mar 20 '13 at 12:51
  • I don't follow your first sentence sorry -- the *best* case would be if L0 and L1 have the same number of points, as it may then be possible to match each point in one with some distinct point in the other. Re sentence 2: There is weighted bipartite maximum matching. This might be preferable, as you could give higher weightings to points that were closer to epipolar lines. – j_random_hacker Mar 20 '13 at 14:49
  • This is what I have at all times. L0 and L1 always have the same number of points, and each point from L0 matches one point in L1. – user2187623 Mar 20 '13 at 15:35