4

I'm currently working on a project that needs the shape of objects in two images to be compared.

I have to check if the shape varies above a certain threshold.

I already have the ROI(Region Of Interest) where the objects can occur in the images.I just need to compare the shape in that ROI.

Many objects may be present in the ROI(objects position may vary) and I have to check if they are also present at approximately the same spot on the second image by comparing the shapes. In short, I need to check for similarity between the shapes present in the two images.

  • I tried contours, but it doesn't work well in finding all contours and varies with different images.
  • I am now trying to use Histogram of Oriented Gradients(HOG) approach now. I have got the feature vectors for two images separately. But I don't know how to compare the HOG feature vectors of the two images to find shape similarity.
  • Deep learning and CNN can only be used if the object to be detected is known beforehand along with a large number of training images. My problem is that object can be anything and I don't have large training sets. So, I decided to go with shape descriptors like HOG, SIFT, SURF,... But I don't know how to work with them effectively to get desired outputs.
  • I can't understand the outputs(feature vectors) from the descriptors. I don't know how to use feature vectors from both the images to compare the shapes present in the two images.

Can anyone help me in understanding and working with shape descriptors(HOG, SIFT, SURF,...)? How can we compare the HOG feature vectors of two images? or is there a better way to compare if two images have similar shapes in them? Detecting and Comparing Shapes is the goal.

Sakthi Geek
  • 391
  • 4
  • 10
  • You could use an edge detection algorithm as in here http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_canny/py_canny.html – Pumpkin Dec 04 '17 at 11:17
  • I have already tried Canny Edge Detector, but the output varies with different images with different lighting. It doesn't detect edges consistently to be used in shape detection and comparison. How can I compare the shapes in two images with edge detection? I only know an ROI within which any object can be present. I have to find if it's the same object or not by comparing shapes. – Sakthi Geek Dec 04 '17 at 11:34

1 Answers1

2

Detecting and Comparing Shapes between two images can be broken into two parts.

  • Detecting Shapes

  • Comparing Shapes

Detecting Shapes

Detecteting Shapes can be easy tackled using the OpenCV library available. (https://docs.opencv.org/trunk/d4/dc6/tutorial_py_template_matching.html)

import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('messi5.jpg',0)
img2 = img.copy()
template = cv2.imread('template.jpg',0)
w, h = template.shape[::-1]
# All the 6 methods for comparison in a list    
methods = ['cv2.TM_CCOEFF', 'cv2.TM_CCOEFF_NORMED', 'cv2.TM_CCORR',
        'cv2.TM_CCORR_NORMED', 'cv2.TM_SQDIFF', 'cv2.TM_SQDIFF_NORMED']
for meth in methods:
    img = img2.copy()
    method = eval(meth)
    # Apply template Matching
    res = cv2.matchTemplate(img,template,method)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
    # If the method is TM_SQDIFF or TM_SQDIFF_NORMED, take minimum
    if method in [cv2.TM_SQDIFF, cv2.TM_SQDIFF_NORMED]:
        top_left = min_loc
    else:
        top_left = max_loc
    bottom_right = (top_left[0] + w, top_left[1] + h)
    cv2.rectangle(img,top_left, bottom_right, 255, 2)
    plt.subplot(121),plt.imshow(res,cmap = 'gray')
    plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
    plt.subplot(122),plt.imshow(img,cmap = 'gray')
    plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
    plt.suptitle(meth)
    plt.show()

Comparing Shapes

The match template can also give scores of matching and the scale, which could be used to compare shapes.

The only caveat would be that the score is not relative and depends on the image size and the matching algorithms used. This needs to be tweaks according to the need.

Fakabbir Amin
  • 989
  • 7
  • 16
  • Thanks, Fakabbir. But the thing is I don't have the template of the shape to be used for template matching. Many objects may be present in the ROI and I have to check if they are also present at approximately the same spot on the second image by comparing the shapes. In short, I need to check for similarity between various shapes present in the two images. – Sakthi Geek Dec 05 '17 at 05:32
  • Maybe you should look at the following: https://stackoverflow.com/questions/44785958/opencv-detect-changes-between-two-photos-taken-by-different-time – Fakabbir Amin Dec 05 '17 at 05:59
  • I checked out the above link. My problem can also be stated as images taken at a different time with changes in object position, lighting and presence. But the solution that was given is based on pixel by pixel comparison without considering shapes at all and doesn't work with the images with positional and intensity change. – Sakthi Geek Dec 05 '17 at 13:11
  • So your problem boils to the following point that, given two images you have to detect an object and compare shape. Had you looked at the following post ? https://www.pyimagesearch.com/2017/09/11/object-detection-with-deep-learning-and-opencv/ – Fakabbir Amin Dec 06 '17 at 11:08
  • The example in the post uses a deep learning to do the job. This works only if the object to be detected is known and if we have a large set of training images for those objects. But I don't have both. So only I decided to go with shape descriptors like HOG, SIFT, SURF,...(refer the question above). But I don't understand the outputs from these. I don't know how I can use the feature vectors effectively to compare the shapes in both images. Do you have knowledge in this area? – Sakthi Geek Dec 06 '17 at 13:10
  • I have edited the question adding more details. Please check it out and help me if you know something related to that. – Sakthi Geek Dec 06 '17 at 13:36