Real time feature matching with stereo camera

Question

I have a stereo camera setup and I am trying to match features between the two frames so that I can triangulate the corresponding points into a 3d point cloud. It kind of works, using SURF, but is too slow for real-time use. Is there a faster way? Or, a way around the problem?

This is my code:

bool matchFeatures(Mat img_1, Mat img_2)
{   
    points_2D_left.clear();
    points_2D_right.clear();
    //-- Step 1: Detect the keypoints using SURF Detector
    int minHessian = 400;   SurfFeatureDetector detector(minHessian);

    std::vector<KeyPoint> keypoints_1, keypoints_2;

    detector.detect(img_1, keypoints_1);
    detector.detect(img_2, keypoints_2);

    //-- Step 2: Calculate descriptors (feature vectors)
    SurfDescriptorExtractor extractor;
    Mat descriptors_1, descriptors_2;

    extractor.compute(img_1, keypoints_1, descriptors_1);
    extractor.compute(img_2, keypoints_2, descriptors_2);

    //-- Step 3: Matching descriptor vectors using FLANN matcher

    FlannBasedMatcher matcher;
    std::vector< DMatch > matches;
    matcher.match(descriptors_1, descriptors_2, matches);

    double max_dist = 0; double min_dist = 100;

    //-- Quick calculation of max and min distances between keypoints
    for (int i = 0; i < descriptors_1.rows; i++)
    {
        double dist = matches[i].distance;
        if (dist < min_dist) min_dist = dist;
        if (dist > max_dist) max_dist = dist;
    }

    std::vector< DMatch > good_matches;

    for (int i = 0; i < descriptors_1.rows; i++)
    {
        if (matches[i].distance <= max(2 * min_dist, 0.02))
        {
            good_matches.push_back(matches[i]);
        }
    }



    for (int i = 0; i < good_matches.size(); i++)
    {
        //-- Get the keypoints from the good matches
        points_2D_left.push_back(keypoints_1[good_matches[i].queryIdx].pt);
        points_2D_right.push_back(keypoints_2[good_matches[i].trainIdx].pt);
    }


    return true;
}

Have you tried to profile your code? where are the bottlenecks? — Ophir Gvirtzer, Jun 01 '15 at 15:41
Hi, it looks like this section: extractor.compute(img_1, keypoints_1, descriptors_1); extractor.compute(img_2, keypoints_2, descriptors_2); — anti, Jun 01 '15 at 16:15

score 1 · Accepted Answer · answered Jun 28 '15 at 10:42

1

SURF is slow. Try to use ORB, which operates in real time. OrbFeatureDetector

answered Jun 28 '15 at 10:42

user3678464

46
2

Real time feature matching with stereo camera

1 Answers1