2

I want to compute similarities between two images using SIFT. I have managed to compute matches and visualize it as seen in the image below. Sift between heavily rotated Eiffel tower and normal Eiffel tower. I have one image of the Eiffel tower and another image of a heavily modified Eiffel tower. To me this match looks good but I don't know what metrics, equations or algorithms to use to compute the similarity or to evaluate the match.

I am using the following code to compute the matching.

import cv2

# Read images
img1 = cv2.imread("eiffel_normal.jpeg")
img2 = cv2.imread("eiffel_rotated.jpeg")

#sift
sift = cv2.SIFT_create()

# Get keypoints and descriptors
keypoints_1, descriptors_1 = sift.detectAndCompute(img1, None)
keypoints_2, descriptors_2 = sift.detectAndCompute(img2, None)

#feature matching
bf = cv2.BFMatcher(cv2.NORM_L1, crossCheck=True)

matches = bf.match(descriptors_1,descriptors_2)
matches = sorted(matches, key=lambda x:x.distance)

# Visualize the results
img3 = cv2.drawMatches(img1, keypoints_1, img2, keypoints_2, matches[:30], img2, flags=2)
plt.imshow(img3)
plt.show()

I've tried:

def calculateScore(matches, key_1_len, key_2_len):
    return 100 * (matches/min(key_1_len, key_2_len))

similar_regions = [i for i in matches if i.distance < 50]
sift_score= calculateScore(len(matches), len(keypoints_1), len(keypoints_2))
sift_acc = len(similar_regions)/len(matches)

but both sift_score and sift_acc gives bad results.

The evaluator must take in account: Scaling, Rotation and translation

Any ideas?

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Oskar Tengberg
  • 41
  • 5
  • 1
    be aware that soliciting opinions doesn't fit the format of Stack Overflow. -- don't just ignore all but the best 30 matches. use them all. -- use the *number of matches* as a measure. -- look up **Lowe's ratio test**. **do not** use a *fixed* distance. it's highly dependent on the type of descriptor! consider the number of inliers and good matches, vs number of keypoints in each image. -- warp one image into the other, consider only the overlap, calculate sum of absolute per-pixel differences or something -- judge the homography matrix for "severity" – Christoph Rackwitz Oct 08 '21 at 19:29
  • also https://en.wikipedia.org/wiki/Content-based_image_retrieval – Christoph Rackwitz Oct 08 '21 at 19:32
  • It was not my intention to misuse Stack Overflow, I'm new to this forum. Could you perhaps let me know how I solicited opinions so I won't repeat that mistake in the future? How can the distance alone be used when the images might be rotated, translated and scaled and even photographed with an angle? How do I warp one of the image into the other? I can ofc do this using Photoshop but I need my script to do evaluate the similarity of the images. – Oskar Tengberg Oct 11 '21 at 15:00
  • SO is intended for rather narrow issues of programming, rather than scientific discussions. you didn't really, but this is a broad topic with plenty of approaches. a review of "the literature", even textbooks, would reveal the most well known ones. that type of question can become tedious because the answers will amount to "read a book" or someone regurgitates the usual approaches from... a book. I am not sure your understanding of "distance" matches what it means in this context. it's a measure in "descriptor space". descriptors are designed to be robust/invariant to those transformations. – Christoph Rackwitz Oct 11 '21 at 15:23
  • "How do I warp one of the image into the other?" you don't ask that until you've googled it and searched hard and not one of the many resources that discuss it made sense. that's one of the basic rules on this site. the answer to that is in the official opencv docs... and every time the answer is that trivial, it annoys people. nobody here has got time to hold a lecture every time someone asks. – Christoph Rackwitz Oct 11 '21 at 15:27
  • Understandable. I will look further into the documentation. Thank you for being patient. – Oskar Tengberg Oct 12 '21 at 07:43

0 Answers0