SIFT Object Matching in Python

Question

I'm trying to follow this tutorial with my own images.However, the results I get are not exeacty what I'd exepct. Am I missing something here, or is SIFT just not a good enough solution in this case? Thanks alot.

import numpy as np
import cv2
from matplotlib import pyplot as plt

MIN_MATCH_COUNT = 10

img1 = cv2.imread('Q/IMG_1192.JPG', 0)          # queryImage
img2 = cv2.imread('DB/IMG_1208-1000.jpg', 0) # trainImage

# Initiate SIFT detector
sift = cv2.xfeatures2d.SIFT_create()

# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)



FLANN_INDEX_KDTREE = 0
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks = 50)

flann = cv2.FlannBasedMatcher(index_params, search_params)

matches = flann.knnMatch(des1, des2, k=2)

# store all the good matches as per Lowe's ratio test.
good = []
for m,n in matches:
    if m.distance < 0.7*n.distance:
        good.append(m)


if len(good)>MIN_MATCH_COUNT:
    src_pts = np.float32([ kp1[m.queryIdx].pt for m in good ]).reshape(-1,1,2)
    dst_pts = np.float32([ kp2[m.trainIdx].pt for m in good ]).reshape(-1,1,2)

    M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)
    matchesMask = mask.ravel().tolist()

    h,w, = img1.shape
    pts = np.float32([ [0,0],[0,h-1],[w-1,h-1],[w-1,0] ]).reshape(-1,1,2)
    dst = cv2.perspectiveTransform(pts,M)

    img2 = cv2.polylines(img2,[np.int32(dst)],True,255,3, cv2.LINE_AA)

else:
    print ("Not enough matches are found - %d/%d" % (len(good),MIN_MATCH_COUNT))
    matchesMask = None

draw_params = dict(matchColor = (0,255,0), # draw matches in green color
                   singlePointColor = None,
                   matchesMask = matchesMask, # draw only inliers
                   flags = 2)

img3 = cv2.drawMatches(img1,kp1,img2,kp2,good,None,**draw_params)

plt.imshow(img3, 'gray'),plt.show()

Original images :

And the result:

Answer 1

From the images provided I figured out that SIFT won't figure out key features for images that are extremely big. Consider the individual car image, it is 1728 pixels in width and 2304 pixels in height. This is to big. The other image has a pretty normal size with the car occupying a smaller region.

The certain features expected to be matched would be the rims on the wheels, corners on the windows, corners around the bonnet, etc. But in an enlarged image such as the one provided there are no distinct corners, instead there are more of edges present. SIFT looks out for features points that are distinct in nature (with corners in particular).

After resizing the car image to dimension (605 x 806) and the other image to dimension (262 x 350), there was one correct match found in the following figure (notice the match near the wheel):

I tried out the same code for another bunch of images having some letters and drawings in it. Here is the result for that:

Answer 2

In order to evaluate if the issue come from SIFT descriptor I would advise you use another descriptor such as cv2.xfeatures2d.VGG_create() or cv2.BRISK_create(). Take also a look to cv2.xfeatures2d.matchGMS it may give much better results even with SIFT descriptor.

From my personal experience among the possible reasons that can justify the lack of accuracy in the context of your application of the SIFT algorithm would be the sensitivity to gradient reversal. SIFT descriptor does contains a quantified histogram of the normalised gradient orientation surrounding a keypoint. The thing is if the intensity in a region move from a lighter to a darker pixel (e.g. 255->127) the gradient orientation will be different than if the intensities move from a darker to lighter (e.g.127->255).