How to start with landmark detector for custom objects?

Question

I am trying to make a landmark detector for a custom object, this panel:

I am going through multiple approaches to it: using heatmaps and regression-based ones.

I tried hands on this Kaggle notebook and found it useful, but has data in grayscale and may not work well as the panel is dark itself.

Is there a well-documented method to do so ?

How much data do you have? What is the purpose of landmark detector? Is it for localization, pose estimation? — burak akdemir, Sep 24 '20 at 19:00
I am getting data from google images. They are RGB. I am trying to make a landmark detection that detects the coordinates of the four corners: `top_left`, `top_right`, `bottom_right` and `bottom_left` in strictly clockwise manner. I trained it using the kaggle notebook but don't get good results. :( — Pe Dro, Sep 25 '20 at 10:58

score 1 · Answer 1 · answered Sep 26 '20 at 16:56

If your purpose is to detect the coordinates of the corners than The methods that you proposed are overkill. You can simply do template matching for each of the corners. Refer to the code below for a rough example

import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt
#%%
src = cv.imread('airMH.jpg')

gray = cv.cvtColor(src,cv.COLOR_BGR2GRAY)
blur=cv.GaussianBlur(gray,(3, 3),1)
templateRightBot = blur[170:205,110:135];
templateLeftBot = blur[150:185,15:50];
templateRightTop = blur[35:75,135:160];
templateLeftTop = blur[30:50,80:105];

#%%
img=src.copy()
w, h = templateLeftTop .shape[::-1]
res = cv.matchTemplate(gray, templateLeftTop, cv.TM_CCOEFF)
min_val, max_val, min_loc, max_loc = cv.minMaxLoc(res)
top_left = max_loc
templateLeftTop = (top_left[0] + w, top_left[1] + h)
cv.rectangle(img,top_left, templateLeftTop, 255, 2)
plt.subplot(2,2,1),plt.imshow(img,cmap = 'gray')
plt.title('LeftTop'), plt.xticks([]), plt.yticks([])

img=src.copy()
w, h = templateRightTop .shape[::-1]
res = cv.matchTemplate(gray, templateRightTop, cv.TM_CCOEFF)
min_val, max_val, min_loc, max_loc = cv.minMaxLoc(res)
top_left = max_loc
templateRightTop = (top_left[0] + w, top_left[1] + h)
cv.rectangle(img,top_left, templateRightTop, 255, 2)
plt.subplot(2,2,2),plt.imshow(img,cmap = 'gray')
plt.title('RightTop'), plt.xticks([]), plt.yticks([])

img=src.copy()
w, h = templateLeftBot .shape[::-1]
res = cv.matchTemplate(gray, templateLeftBot, cv.TM_CCOEFF)
min_val, max_val, min_loc, max_loc = cv.minMaxLoc(res)
top_left = max_loc
templateLeftBot = (top_left[0] + w, top_left[1] + h)
cv.rectangle(img,top_left, templateLeftBot, 255, 2)
plt.subplot(2,2,3),plt.imshow(img,cmap = 'gray')
plt.title('LeftBot'), plt.xticks([]), plt.yticks([])

img=src.copy()
w, h = templateRightBot .shape[::-1]
res = cv.matchTemplate(gray, templateRightBot, cv.TM_CCOEFF)
min_val, max_val, min_loc, max_loc = cv.minMaxLoc(res)
top_left = max_loc
templateRightBot = (top_left[0] + w, top_left[1] + h)
cv.rectangle(img,top_left, templateRightBot, 255, 2)
plt.subplot(2,2,4),plt.imshow(img,cmap = 'gray')
plt.title('RightBot'), plt.xticks([]), plt.yticks([])

here is the output:

about converting image into grayscale, since your images are taken outdoors, you need to normalize them to get rid of the effects of sunlight, etc... so converting images into grayscale kind for serves as normalization. So it does more good than harm.

Thanks for this @burak, but unfortunately, the image does not represent all the images/scenes that the camera will see. Template matching is for detecting **same object(s) in the image**. — Pe Dro, Sep 27 '20 at 04:23
Then maybe you can try SIFT features as explained below https://docs.opencv.org/master/da/df5/tutorial_py_sift_intro.html Extracting features then training 4 classifiers (using svm) may be a good solution. You can also use pretrained vgg16 to extract the features. — burak akdemir, Sep 27 '20 at 07:45

How to start with landmark detector for custom objects?

1 Answers1