1

Im using openCV to do template matching. Im planning to use it to detect templates from a live feed. Currently Im working with 63 templates and the total time of matching averages at about 9ms for an image. This is too slow for the application i have in mind.

Optimization Ive done so far:

  1. Grayscale everything
  2. Cropping the source image
  3. Downsampling everything

What else could I try to cut the time down? Im thinking that the feed gonna be about 20fps, and Ill have more CV stuff running along this, so something like 1ms would be the optimal goal.

Heres my code so far:

import cv2 as cv
import numpy as np
import os
import time as time

scale=0.8

#load needles
directory = r"FOLDER_OF_NEEDLES"
needleList = []
for fails in os.listdir(directory):
    path = os.path.join(directory, fails)
    img  = cv.imread(path, cv.IMREAD_GRAYSCALE)
    img_resized = cv.resize(img, None, fx=scale, fy=scale, interpolation = cv.INTER_AREA)
    needleList.append(img_resized)
    
#load haystack
haystack_img = cv.imread('HAYSTACK_IMAGE', cv.IMREAD_UNCHANGED)
haystack_img_gray = cv.cvtColor(haystack_img, cv.COLOR_BGR2GRAY)
shop_haystack = haystack_img_gray[668:768, 125:900]
shop_haystack = cv.resize(shop_haystack,None, fx=scale, fy=scale, interpolation = cv.INTER_AREA)

#debugging stuff
how_many_matches = 0
timeStart = time.time()

threshold = 0.85

#matching loop
for needle in needleList:
    result = cv.matchTemplate(shop_haystack, needle, cv.TM_CCOEFF_NORMED)
    min_val, max_val, min_loc, max_loc = cv.minMaxLoc(result)
    
#debugging stuff
    #print('Best match top left position: %s' % str(max_loc))
    #print('Best match confidence: %s' % max_val)

#draw rectangles on matches    
    if max_val >= threshold:
        how_many_matches+=1
      
        needle_h = int(needle.shape[0]/scale)
        needle_w = int(needle.shape[1]/scale)
        top_left = max_loc
        top_left = (
            int(top_left[0]/scale+125),
            int(top_left[1]/scale+668))
        bottom_right = (top_left[0] + needle_w, top_left[1] + needle_h)

        cv.rectangle(
            haystack_img,
            top_left,
            bottom_right,
            color = (0, 255, 0),
            thickness = 2,
            lineType = cv.LINE_4)

print(time.time()-timeStart)
print(how_many_matches, 'matches')
cv.imshow('slowboi', haystack_img)
cv.waitKey()

I have provided the needles here: https://failiem.lv/u/t6sqtx7tv

And a couple haystacks here: https://failiem.lv/u/bttrwy6mc

Any and all help would be appreciated!

Pēteris Andrušaitis
  • 23
  • 3
  • this looks like the "needles" only appear in a very small set of exact locations (five to be exact), not at arbitrary pixel positions in a larger picture... matchTemplate is the wrong tool. at best, you should give matchTemplate an exact-same-size ROI and use it merely for calculating the matching score for that exact position. – Christoph Rackwitz Jul 16 '21 at 00:36
  • So youre basically saying that I should restrict the source image further? You mentioned matchTemplate to be the wrong tool, would you suggest as the right tool? – Pēteris Andrušaitis Jul 16 '21 at 06:44
  • Assuming two arrays (`ref` and `sample`) of equal size, you could do detection based on low or lowest mean square error: `mse = np.mean((ref.flatten()-sample.flatten())**2)` – Bart van Otterdijk Jul 25 '21 at 22:15

0 Answers0