1

I use two cameras to simultaneously take pictures of the same scenery. The two resulting pictures have a different aspect ratio and are slightly offset, as the cameras are positioned a few centimeters apart. My goal is to produce an overlay of both images.

I'm using OpenCV python with SURF for feature detection and FLANN for feature matching. To sort out false matches I use Lowe's ratio test and RANSAC.

import numpy as np
import cv2
from matplotlib import pyplot    

image_path1 = "./images/IMG01.jpg"
img1 = cv2.imread(image_path1, 0)
image_path2 = "./images/IMG02.jpg"
img2 = cv2.imread(image_path2, 0)
    
# Initiate SURF detector
surf = cv2.xfeatures2d.SURF_create()

# find the keypoints with SURF
kp1, des1 = surf.detectAndCompute(img1, None)
kp2, des2 = surf.detectAndCompute(img2, None)
    
# FLANN matcher
index_params = dict(algorithm = 1, trees = 5)
search_params = dict()

flann = cv2.FlannBasedMatcher(index_params,search_params)

matches = flann.knnMatch(np.float32(des1), np.float32(des2),k=2)

# Need to draw only good matches, so create a mask
matchesMask = [[0,0] for i in range(len(matches))]

# ratio test as per Lowe's paper
good = []
for i,(m,n) in enumerate(matches):
    if m.distance < 0.5*n.distance:
        good.append(m)

MIN_MATCH_COUNT = 5

if len(good) > MIN_MATCH_COUNT:
    src_pts = np.float32([ kp1[m.queryIdx].pt for m in good ]).reshape(-1,1,2)
    dst_pts = np.float32([ kp2[m.trainIdx].pt for m in good ]).reshape(-1,1,2)
    M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5)
    matchesMask = mask.ravel().tolist()
else:
    print( "Not enough matches are found - {}/{}".format(len(good), MIN_MATCH_COUNT) )
    matchesMask = None

draw_params = dict(matchColor = (0,255,0),
                   singlePointColor = (255,0,0),
                   matchesMask = matchesMask,
                   flags = 0)

img3 = cv2.drawMatches(img1,kp1,img2,kp2,good,None,**draw_params)

pyplot.imshow(img3,),pyplot.show()

Sometimes, e.g. for low light images, there are very few correct matches (< 10). Thus the homography is not accurate (as described here).

Is there an alternative approach, knowing that the images are just slightly offset?

erkor
  • 57
  • 1
  • 5
  • you may use other feature detector descriptor and matcher framework to get more features. Search things like ORB, KAZE – Dr Yuan Shenghai Nov 24 '20 at 02:32
  • 2
    Please show input image examples. Have you considered template matching? or cv2.findTransformECC() with cv2.MOTION_EUCLIDEAN and cv2.warpAffine – fmw42 Nov 24 '20 at 06:27
  • Thank you for your help! Surprisingly SURF worked out the best for my images. I tried ORB and KAZE/AKAZE beforehands. @fmw42 cv2.findTransformECC() actually worked really well and I'm currently reworking my algorithm based on it. – erkor Nov 26 '20 at 13:43

0 Answers0