OpenCV 2.2 SURF Feature matching problems

Question

Matching with nothing in the top right corner

I have modified the OpenCV demo application "matching_to_many_images.cpp" to query a image (left) to a frames from the webcam (right). What have gone wrong with the top right corner of the first image?

We think this is related to another problem we have. We begin with an empty database and we only add unique (features that not match the features in our database) but after adding only three features, we get a match on all new features....

we are using: SurfFeatureDetector surfFeatureDetector(400,3,4); SurfDescriptorExtractor surfDescriptorExtractor; FlannBasedMatcher flannDescriptorMatcher;

Complete code can be found at: http://www.copypastecode.com/71973/

To be 100% correct, it is the top-LEFT corner of the RIGHT image. We need a way to sift through these obviously false matches, any ideas? Why are these points detected as matches? — Orka, May 30 '11 at 09:37
You should mark my answer as the correct one. The actual one is misleading. — Diego Cerdan Puyol, May 12 '13 at 12:28

KMS · Accepted Answer · 2011-05-31T06:09:07.113

I think this has to do with the border keypoints. The detector detects the keypoints, but for the SURF descriptor to return consistent values it needs pixel data in a block of pixels around it, which is not available in the border pixels. You can use the following snippet to remove border points after keypoints are detected but before descriptors are computed. I suggest using borderSize of 20 or more.

removeBorderKeypoints( vector<cv::KeyPoint>& keypoints, const cv::Size imageSize, const boost::int32_t borderSize )
{
    if( borderSize > 0)
    {
        keypoints.erase( remove_if(keypoints.begin(), keypoints.end(),
                               RoiPredicatePic((float)borderSize, (float)borderSize,
                                            (float)(imageSize.width - borderSize),
                                            (float)(imageSize.height - borderSize))),
                     keypoints.end() );
    }
}

Where RoiPredicatePic is implemented as:

struct RoiPredicatePic
{
    RoiPredicatePic(float _minX, float _minY, float _maxX, float _maxY)
    : minX(_minX), minY(_minY), maxX(_maxX), maxY(_maxY)
    {}

    bool operator()( const cv::KeyPoint& keyPt) const
    {
        cv::Point2f pt = keyPt.pt;
        return (pt.x < minX) || (pt.x >= maxX) || (pt.y < minY) || (pt.y >= maxY);
    }

    float minX, minY, maxX, maxY;
};

Also, approximate nearest neighbor indexing is not the best way to match features between pairs of images. I would suggest you to try other simpler matchers.

Neat, good point! In our "production" code we have switched to using the brute force matcher instead. I'll look into sorting out these points with your code snippet as a guide! — Orka, May 31 '11 at 06:15
This solved the problem that we had before, and answered the question. Sadly this was not the cause of our database-matching problems.. well back to debugging! Thanks for your answer! — Maidenone, May 31 '11 at 10:00

score 4 · Answer 2 · answered Dec 09 '11 at 16:26

Your approach is working flawless but it shows wrong results because of calling incorrectly the drawMatches function.

Your incorrect call was something like this:

drawMatches(image2, image2Keypoints, image1, image1Keypoints, matches, result);

The correct call should be:

drawMatches(image1, image1Keypoints, image2, image2Keypoints, matches, result);

score 3 · Answer 3 · answered Feb 26 '12 at 19:07

I faced the same problem. Surprisingly, the solution has nothing to do with border points or the KNN matcher. Just needs a different matching strategy for filtering out "good matches" from the plethora of matches.

Use a 2 NN search, and the following condition-

if distance(1st match) < 0.6*distance(2nd match) the 1st match is a "good match".

Filter out all the matches that do not satisfy the above condition and call drawMatches for only the "good matches". Voila!

OpenCV 2.2 SURF Feature matching problems

3 Answers3