Use matchTemplate with colored images OpenCV

Question

I am trying to detect certain COLORED images in a desktop screenshot, where I have equally shaped templates but different color (these are not distinguished from one another doing the normal matchTemplate method as it is done with greyscale images) here is the main code that does the detection:

     template = cv2.imread(template_path,1)
    #template_hsv = cv2.cvtColor(template, cv2.COLOR_RGB2HSV)
    #template_B, template_G, template_R = cv2.split(template)
    #scr_B, scr_G, scr_R = cv2.split(screenshot)
    scr_B = screenshot[:, :, 0]
    scr_G = screenshot[:, :, 1]
    scr_R = screenshot[:, :, 2]
    template_B = template[:, :, 0]
    template_G = template[:, :, 1]
    template_R = template[:, :, 2]
    #cv2.imwrite('./sc4.png', scr_R)
    #cv2.imwrite('./template.png', template)

    resB = cv2.matchTemplate(scr_B, template_B, cv2.TM_CCOEFF_NORMED)
    resG = cv2.matchTemplate(scr_G, template_G, cv2.TM_CCOEFF_NORMED)
    resR = cv2.matchTemplate(scr_R, template_R, cv2.TM_CCOEFF_NORMED)

    res = resB + resG + resR

    #res = cv2.matchTemplate(screenshot, template_G, cv2.TM_CCOEFF_NORMED)
    matches = np.where(res >= 3*threshold)
    print(matches)
    return matches

As you can see I tried splitting the channels of the rgb screenshot image, and compare then with the also splitted template image. I also tried doing this with HSV channels as you can see in the commented code. However this did not work and despite seeing that in single channeled images there was a visual difference of color, the program did not distinguish them (I also tried doing comparisons with each individual channel of the template and screenshot) .

All sugestions are welcome, even trying to achieve my goal using anything else. Thank you in advance.

TM_CCOEFF_NORMED will not work with constant color images, because the denominator contains effectively the standard deviation, which is zero for a constant image. — fmw42, Apr 26 '19 at 19:47

score 2 · Answer 1 · answered Apr 24 '19 at 12:51

I tried it with TM_CCOEFF_NORMED method and did not work... somehow it gave everything as 1.0 (all max value).... but with the TM_SQDIFF_NORMED method actually gives something reasonable.

Lets start by creating a sample image, I did it as following:

import numpy as np
import cv2

randVal = lambda : np.random.randint(0,high=255, dtype=np.uint8)
randomColor = lambda : (randVal(), randVal(), randVal())
targetColor = randomColor()

width = 500
height = 500
x = 20
y = 20
img = np.zeros((height, width,3), dtype=np.uint8)
target = np.full((100, 100,3), targetColor, dtype=np.uint8)
while y < height-100:
  x  = 20
  while x < width-100:
    img[y:y+100, x:x+100] = randomColor()
    x += 120
  y += 120

img[20:120, 20:120] = targetColor

This will create 2 images, img and target which are both random and in my test it gives something like this:

img:

target:

Now, I used the template matching as is, since in the documentation says that it may take 1 or 3 channels and it will do it correctly, but with the other method

res = cv2.matchTemplate(img[:,:,0], target[:,:,0], cv2.TM_SQDIFF_NORMED )
threshold = 0.00001
loc = np.where( res <= threshold )
img_rgb = img.copy()
for pt in zip(*loc[::-1]):
    cv2.rectangle(img_rgb, pt, (pt[0] + 100, pt[1] + 100), (0,0,255), 2)

This gives me the following result:

I hope this helps... I need to test the method you use in another version to see if it is a problem in my version or is something specific with this set of images...

can you explain a bit more about the code you provided? i thought both img[:,:,0] and target[:,:,0] would have only one channel which is green of BGR in this case. so is it actually just comparing green value? — kerz, Mar 11 '23 at 05:10
@kerz I guess that this is an error of my part, i think I was trying to use grey scale and only one channel but somehow i ended using the blue channel or maybe the whole image and copied the wrong thing here. I will double check, but certainly it was not what I was intending — api55, Apr 14 '23 at 07:09

Use matchTemplate with colored images OpenCV

1 Answers1