0

I am trying to scan an object with a laser to extract 3D point clouds. There are 2 cameras and 1 laser in my setup. What I do is giving nonzero points in masks to OpenCV's triangulatePoints function as projPoints arg. Since both numbers of points must be the same for triangulatePoints function and there are 2 masks, if one mask has more nonzero points than the other, I basically downsize it to other's size by doing this:

l1 = len(pts1)
l2 = len(pts2)
newPts1 = pts1[0:l2]

Is there a good way for matching left and right frame nonzero points?

Stereo Pain

Mertcan Karık
  • 31
  • 1
  • 7
  • 1
    I think the assumption is wrong, that there must be the same number of points in both images (per line?). If I understand right, you probably only want/need one laser-point per pixel line. You could either use mean/median position or maybe approximate a polynome or a spline? – Micka Jun 30 '21 at 11:43
  • it may be better sharing the frames seperately instead of merged – Yunus Temurlenk Jun 30 '21 at 12:39

1 Answers1

2

First, if your images normally look like that, your sensors are deeply saturated, and consequently your 3D ranges are either worthless or much less accurate than they could be.

Second, you should aim for matching one point per rectified scanline on each image of the pair, rather than a set of points. The whole idea of using a laser stripe is to get a well focused beam of light on as small a spot or band as possible, so you can probe the surface in detail.

For best accuracy, the peak-finding should be done independently on each scanline of the original (distorted and not rectified) images, so it is not affected by the interpolation used by the undistortion and stereo rectification procedures. Rather, you would use the geometrical undistortion and stereo rectification transforms to map the peaks detected in original images into the rectified ones.

There are several classical algorithms for peak-finding with laser stripe-based triangulation methods, you may find this other answer of mine useful.

Last, if your setup is expected to be as in the picture, with the laser stripe illuminating two orthogonal planes in addition to the object of interest, then you do not need to use stereo at all: you can solve for the 3D plane spanned by the laser stripe projector and triangulate by intersecting that plane with each ray back-projecting the peaks of the image of the laser stripe on the object. This is similar to one of the methods J. Y. Bouguet used in his old Ph.D. thesis on desktop photography (here is a summary by S. Seitz). One implementation using a laser striper is detailed in this patent. This method is surprisingly accurate: with it we achieved approximately 0.2mm accuracy in a cubic foot of volume using a dinky 640x480 CCD video camera back in 1999. Patent has expired, so you are free to enjoy it.

Francesco Callari
  • 11,300
  • 2
  • 25
  • 40
  • Thank you for your detailed answer. I will thin my laser, change my cameras, and share results. Can you share more sources about stereo+laser setup for further reading? – Mertcan Karık Jul 01 '21 at 10:36