1

I'm trying to write an app for wild leopard classification and conservation in South Asia. For this, I have the main challenge to identify the leopards by their spot pattern in the forehead.

The current approach I am using is,

  1. Store the known leopard forehead images as a base list
  2. Get the user-provided leopard image and crop the forehead of the leopard
  3. Pre-process the images with the bilateral filter to reduce the noise
  4. Identify the keypoints using the SIFT algorithm
  5. Use FLANN matcher to get KNN matches
  6. Select good matches based on the ratio threshold

Sample code:

# Pre-Process & reduce noise.
img1 = cv.bilateralFilter(baseImg, 9, 75, 75)
img2 = cv.bilateralFilter(userImage, 9, 75, 75)

detector = cv.xfeatures2d_SIFT.create()
keypoints1, descriptors1 = detector.detectAndCompute(img1, None)
keypoints2, descriptors2 = detector.detectAndCompute(img2, None)

# FLANN parameters
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)
search_params = dict(checks=50)  # or pass empty dictionary
matcher = cv.FlannBasedMatcher(index_params, search_params)

knn_matches = matcher.knnMatch(descriptors1, descriptors2, 2)

allmatchpointcount = len(knn_matches)
ratio_thresh = 0.7
good_matches = []

for m, n in knn_matches:
    if m.distance < ratio_thresh * n.distance:
        good_matches.append(m)


goodmatchpointcount = len(good_matches)
print("Good match count : ", goodmatchpointcount)
matchsuccesspercentage = goodmatchpointcount/allmatchpointcount*100
print("Match percentage : ", matchsuccesspercentage)

Problems I have with this approach:

  1. The method has a medium-low success rate and tends to break when there is a new user image.

  2. The user images are sometimes taken from different angles where some key patterns are not visible or warped.

  3. The user image quality affects the match result significantly.

I appreciate any suggestions to get this improved in any manner.

Sample Images

Base Image

Base image

Above is matching to below: (Incorrect pattern matched)

Target image

More sample images as requested.

William Miller
  • 9,839
  • 3
  • 25
  • 46
debugger89
  • 2,108
  • 2
  • 14
  • 16
  • 1
    This is quite a difficult task for feature matching since - as you mentioned - the patterns are distorted and/or not visible. Secondly, there are repetitive patterns which hamper the feature matching. As a starting point I would try to use different feature detectors like e.g. MSER. You could try brute-force matching instead of FLANN if your application is not time-critical and different variations of RANSAC (MSAC, DEGENSAC) for improved outlier removal. – Grillteller Mar 24 '20 at 16:10
  • @Grillteller a description would be nice to understand the difference between the current approach and the proposed. what kinds of difference that would make? I would like to learn more about the topic. – debugger89 Mar 31 '20 at 11:25
  • 2
    A typical method would be to recognize the animal eyes and nose (http://blog.dlib.net/2016/10/hipsterize-your-dog-with-deep-learning.html) and find a rigid transform that normalizes eye-level and width. Then you would select the forehead region-of-interest based on the eye-level and run a feature extractor network to yield an N-length feature descriptor. Finally, you'd match pairs using Cosine distance between feature descriptors. – mainactual Apr 01 '20 at 13:48
  • For human face, see OpenVino sample https://docs.openvinotoolkit.org/2018_R5/_docs_Retail_object_reidentification_face_mobilenet_based_onnx_0095_desc_face_reidentification_retail_0095.html – mainactual Apr 01 '20 at 13:48
  • Could you add some more images and/or your results? I would try some algorithms but one image pair is not enough to tell if there exists a general solution :) – Grillteller Apr 01 '20 at 15:03
  • @mainactual, the first part of your suggestion is already covered by a separate process. For identification, I always get these two images of the forehead perfectly cropped. What I'm struggling with is the matching part. Can you please explain more about that? – debugger89 Apr 03 '20 at 13:15
  • @Grillteller, I have uploaded multiple images as you requested. https://drive.google.com/open?id=1bD17SGhpNSwlQWaipOwu1kBc9Djo32Xu – debugger89 Apr 03 '20 at 13:48
  • 1
    LBP is known to describe patterns very well. I suggest you consider LBP + uniform histograms + pca + svm. I worked on a similar issue and pretty sure it is doable for your case. The configuration of the "pipeline" depends on many things, for example, how many train images you have and so on. Can be done without svm as well but less flexible. Did you see this? https://www.pyimagesearch.com/2015/12/07/local-binary-patterns-with-python-opencv/ – Cynichniy Bandera Apr 03 '20 at 16:29

0 Answers0