Opencv matchTemplate couldn't find bush inside map

Question

I want to find 2d images of bush inside a 2d map. According to their website I used this code

image = cv2.imread('images/Pallet_Town.png', cv2.IMREAD_UNCHANGED)
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
template = cv2.imread('images/Bush (small).png', cv2.IMREAD_UNCHANGED)

template_gray = cv2.cvtColor(template, cv2.COLOR_BGR2RGB)

print(image.shape)
w, h, _ = template.shape

result = cv2.matchTemplate(image, template, cv2.TM_CCOEFF_NORMED)
threshold = 0.8
loc = np.where(result >= threshold)
print(*loc[::-1])
for pt in zip(*loc[::-1]):
    cv2.rectangle(image, pt, (pt[0] + w, pt[1] + h), (0, 0, 255), 2)
cv2.imwrite('images/result.png', image)

The code doesn't work and is not able to find any bush. The result image has no rectangles. What else function should I use to find the bush inside the image (Be it an opencv function or any other package) or is it my code problem.

Using these images (Bush is under the map)

Bush:

Is the bush template in the right scale? Same size as any bush in the image? The missing background in the bush template might be an issue. — Micka, May 22 '22 at 06:42
@Micka Yes the bush(16x16) is of the same scale in map. Luckily I have found the answer — ROOP AMBER, May 22 '22 at 07:09
use `TM_SQDIFF`. why is everyone using those rubbish modes? is there some youtube tutorial video that tells everyone to do this? — Christoph Rackwitz, May 22 '22 at 12:15
@ChristophRackwitz I only used it because in the opencv docs for Template Matching with Multiple Objects they have used cv2.TM_CCOEFF_NORMED. I just took the code from there (almost) and hence used the same function — ROOP AMBER, May 25 '22 at 16:51
thanks. that's good to know. I found the section you mean. I wish that tutorial would discuss these modes more mathematically. perhaps I should open an issue about that... — Christoph Rackwitz, May 25 '22 at 20:18
@ChristophRackwitz I completely agree I am still not sure which method to use on occasions they have not explained it nicely. Do you know a website that mention the correct usage and explanation — ROOP AMBER, May 26 '22 at 07:08

ROOP AMBER · Accepted Answer · 2022-05-23T16:41:45.997

2

I have actually found the answer, As fmw42 suggested me in comments making a mask of the bush and hence using this code I got the answer. Later I also had to increase the threshold to 0.9. Alternatively reducing threshold to 0.24 also worked but for different images it would have different thresholds which I didn't want

image = cv2.imread('images/Pallet_Town.png', cv2.IMREAD_UNCHANGED)
template = cv2.imread('images/Bush (small).png', cv2.IMREAD_UNCHANGED)

bush = template[:, :, 0:3]
alpha = template[:, :, 3]
alpha = cv2.merge([alpha, alpha, alpha])
result = cv2.matchTemplate(image, bush, cv2.TM_CCORR_NORMED, mask=alpha)
w, h, _ = template.shape

threshold = 0.9
loc = np.where(result >= threshold)
for pt in zip(*loc[::-1]):
    cv2.rectangle(image, pt, (pt[0] + w, pt[1] + h), (0, 0, 255), 2)
cv2.imwrite('images/result.png', image)

edited May 23 '22 at 16:41

answered May 22 '22 at 07:10

ROOP AMBER

330
1
8

2

Your template has a transparent background. But your matchTemplate is not using the mask from the transparency alpha channel. So you are matching with the background from whatever is under the transparency, which likely does not match the area around the bush well. In your template that background is black and your bushes have a white background. So either change the template background to white or use the mask from the alpha channel in the matchTemplate(). See the documentation for matchTemplate for the use of a mask image. – fmw42 May 22 '22 at 14:58
1

See my example at https://stackoverflow.com/questions/67368951/opencv-matchtemplate-and-np-where-keep-only-unique-values/67374288#67374288 and https://stackoverflow.com/questions/61779288/how-to-template-match-a-simple-2d-shape-in-opencv/61780200#61780200 – fmw42 May 22 '22 at 15:00
@fmw42 I couldn't understand how you created a mask of the objects like chess pieces, can you write an answer for my question and explain how I can create a mask for my bush – ROOP AMBER May 23 '22 at 07:45
1

Extract the alpha channel from your template and use that. `mask=bush[:,:,3]` and `bush=bush[:,:,0:3]` so that bush is just the bgr channels. – fmw42 May 23 '22 at 15:08

score 2 · Answer 2 · answered May 23 '22 at 16:54

Your template has a transparent background. But your matchTemplate is not using the mask from the transparency alpha channel. So you are matching with the background from whatever is under the transparency, which likely does not match the area around the bush well. In your template that background is black and your bushes have a white background. So either change the template background to white or use the mask from the alpha channel in the matchTemplate(). See the documentation for matchTemplate for the use of a mask image (last argument in the following) in Python/OpenCV:

result  =   cv.matchTemplate(   image, templ, method[, result[, mask]]  )

So take your template image (the bush):

and extract the alpha channel and keep only the BGR channels for the image:

mask = template[:,:,3]

template = template[:,:,0:3]

Do not convert to grayscale. matchTemplate can work with color images.

Thanks, I understood your answer and wrote my own answering my own question (lol) explaining my new code. Thanks a lot — ROOP AMBER, May 25 '22 at 16:48

Opencv matchTemplate couldn't find bush inside map

2 Answers2

Linked