How to find the index of the "train" image to which the matched keypoint belongs to in Android

Question

After a few hours of research, i still can't find the index of the "train" image to which the matched keypoint belongs to. What i means is

FeatureDetector surfDetector = FeatureDetector.create(FeatureDetector.FAST);

MatOfKeyPoint vector = new MatOfKeyPoint();

surfDetector.detect( mImg, vector );

DescriptorExtractor siftDescriptor =DescriptorExtractor.create(DescriptorExtractor.BRIEF);

Mat descriptors=new Mat();

siftDescriptor.compute(mImg, vector, descriptors);

DescriptorMatcher matcherBruteForce=DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_SL2);

List<MatOfDMatch> matches = new ArrayList<MatOfDMatch>(); 

matcherBruteForce.match(descriptors, descriptors, matches, 2);

I just use the same image as an example.After this,how to find the index of the "train" image to which the matched keypoint belongs to?

Answer 1

I think you are missing something: probably, you try to find a specific object, and you try to find, in a collection of several images, which is the image that has the "best match" with the keypoints of the object you are looking for. Just looking at the sample code you provide, you extract all SIFT/SURF keypoints for an unknown image, and apply the matcher between the object keypoints and your current image. What you need is some kind of metric that tells you how good is the match between your images. The simplest one beeing the count of the number of matched keypoints. Then, you just need to remember which image in your collection has led to the maximum number of matched keypoints. The number of matche keypoints is probably not the best metric to use, and you should check the vast literature on object detection using SIFT and related methods to find one which best suits your purpose.

By the way, your code is quite confusing: you declare a feature detector named surfDetector but instantiate a "FAST" detector. You declare a feature extractor named siftDescriptor, but instantiate a "BRIEF" extractor. I suggest that you keep the same detector and extractor when they exists, e.g. SURF detector/extractor.

Answer 2

I guess this is what you are looking for, just adapt the variable names to your case:

//scene = query; object = train

Point point1 = keypoints_scene.get( matches.get(i).queryIdx ).pt;

Point point2  = keypoints_object.get( matches.get(i).trainIdx ).pt ;

To find the best match you do the following, each match has a parameter called distance. The one with the smallest distance is the best match. So you go throught the list of all your matches and find the one with the smallest distance...

float min_dist = Float.MAX_VALUE;
int min_position = 0;
for( int i=0; i<matches.size(); i++) {
   if( matches.get(i).distance < min_dist ) {
      min_dist = matches.get(i).distance;
      min_position = i;
   }                    
}

Point best_point_scene   = keypoints_scene.get( matches.get(min_position).queryIdx ).pt;
Point best_point_object  = keypoints_object.get( matches.get(min_position).trainIdx ).pt ;