CV::evaluateFeatureDetector doesn't work?

Question

I want to evaluate some detector

#include <opencv.hpp>
#include <opencv2/xfeatures2d.hpp>
#include <iostream>

using namespace std;
using namespace cv;

int main(void)
{
    //===================VARIABLES==========================
    //input images
        Mat img_1 = imread("data/graf1.png", IMREAD_GRAYSCALE);
        Mat img_2 = imread("data/graf3.png", IMREAD_GRAYSCALE);
    //homography matrix
    Mat H1to2;
    FileStorage fs("data/H1to3p.xml", FileStorage::READ);
    fs.getFirstTopLevelNode() >> H1to2;
    //keypoints vector
        vector<KeyPoint> keypoints_1, keypoints_2;
    //decripotrs matrices
    Mat desc1, desc2;
    //SIFT detector
    Ptr<xfeatures2d::SIFT> sift = xfeatures2d::SIFT::create();
    //Descriptor evaluation
    float repeatability;
    int correspCount;
    evaluateFeatureDetector(img_1,img_2,H1to2,&keypoints_1,&keypoints_2,repeatability,correspCount,sift);
    cout<<"repeatability="<<repeatability<<" correspCount="<<correspCount<<" Keypoint 1st="<<keypoints_1.size()<<" Keypoint 2st="<<keypoints_2.size()<<endl;
    return 0;
}

And this is the output:

repeatability=0.484741 correspCount=953 Keypoint 1st=2667 Keypoint 2st=3498

But since the repeatability=correspCount/min(Keypoint 1st,Keypoint 2st) the value above is not correct.

In this answer a possible cause is explained, but looking how evaluateFeatureDetector is implemented in evaluation.cpp (especially calculateRepeatability) it seems that this not-common keypoints are already removed:

...

if( ifEvaluateDetectors )
{
    overlapThreshold = 1.f - 0.4f;

    // remove key points from outside of the common image part
    Size sz1 = img1.size(), sz2 = img2.size();
    filterEllipticKeyPointsByImageSize( keypoints1, sz1 );
    filterEllipticKeyPointsByImageSize( keypoints1t, sz2 );
    filterEllipticKeyPointsByImageSize( keypoints2, sz2 );
    filterEllipticKeyPointsByImageSize( keypoints2t, sz1 );
}
...

score 0 · Answer 1 · answered Jan 16 '17 at 19:22

In evaluation.cpp at line 429:

size_t size1 = keypoints1.size(), size2 = keypoints2t.size();
size_t minCount = MIN( size1, size2 );

where keypoints1 is the actual keypoints in the first image, and keypoints2t is the projection of keypoints in the second image on the first image. In other words, keypoints2t=proj(keypoints2,H^-1). Thus, the denominator in the formula might either come from size(keypoints1) or size(projection of keypoints2).

In the output line you provided, I assume correspCount=953, size1=2667 and size2=1966.

score 0 · Answer 2 · answered Mar 09 '18 at 00:19

The code of EvaluateFeatureDetector() seems to implement exactly the measurements as described in K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool. A comparison of affine region detectors. IJCV, 1(65):43–72, 2005.

They write "We take into account only the regions located in the part of the scene present in both images." Also the normalize the size of the features to a common scale of 30 and accept an overlap error smaller than 40 % as repitition of a feature.

The same criterion is also mentioned in K. Mikolajczyk and C. Schmid, “Scale & affine invariant interest point detectors,” Int. J. Comput. Vis., vol. 60, no. 1, pp. 63–86, 2004. But there they also have the criterion that the centeres of the features must not be further away from each other that 1.5 pixels. That I couldn't find in their implementation, but the paper is also older and maybe they redicided about this criterion.

In case you are still interested you may also check out this website, which will lead you also to the original matlab iplementation, which I guess is used by people.

CV::evaluateFeatureDetector doesn't work?

2 Answers2