0

I am trying to locate QR position squares withing a QR code. QR position squares are the three boxes in the corners of a QR code:

A QR Code positioning block.

I am template matching these to an image provided to me. I have discovered that there are significant 'ghosts' which appear in my output upon template matching:

enter image description here

Note the white region on the left is a giant smear of template, more distinct templates appear in black along the bottom but the whole image is essentially a smear of templates. Here is my code:

import cv2
import numpy as np

qrs=['A','B','C','D','E']
for qr in qrs:
    
    template=cv2.imread(r'C:\Users\Matt\Desktop\Virginia Tech\CV\QR_Template_final.png',0)    
    img = cv2.imread(r'C:\Users\Matt\Desktop\Virginia Tech\CV\QR_'+qr+'.png', 0)
    img_origin = cv2.imread(r'C:\Users\Matt\Desktop\Virginia Tech\CV\QR_'+qr+'.png')
    
    # Binarize
    img[img>125]=255
    img[img<126]=0
    img=img.astype(np.float32)
    template=template.astype(np.float32)

        
    # Template Matching
    res=cv2.matchTemplate(img, template, cv2.TM_CCORR_NORMED)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
    res=255*(res-min_val)/(max_val-min_val)
    
    out=cv2.imwrite(r'C:\Users\Matt\Desktop\Virginia Tech\CV\QR\Threshold\QR_IMG'+qr+'.png',img)
    out=cv2.imwrite(r'C:\Users\Matt\Desktop\Virginia Tech\CV\QR\Threshold\QR_Threshold'+qr+'.png',res)

How do I address these ghosts?

I included a compressed version of the input. It is a picture of a QR code front and center in a grocery store without template any ghosts:

enter image description here

Also see this version of the output without binarization of the input:

enter image description here

Thanks so much for your help I have been stuck on this problem for some time.

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
mfergus9
  • 23
  • 2
  • 1
    What you are seeing in that image is the result of sliding the template through the target image. The brightest locations indicate the highest matches, you should see the points (pixels) close to a value of `255`. These pixels are the best (highest) location of correspondence between both images. Check out the brightest pixels, they should approximately be located at the three corners of the QR Code. – stateMachine Jul 17 '22 at 03:22
  • I think template match is not the best tool for this, though. At least, you are going to need to modify your logic because the `minMaxLoc` function only returns the largest (along with the smallest) location found of all brightest points available. This info is stored in the `max_loc` and `max_value` variables. If you draw a point located at `max_loc` you will see the maximum point of "similarity" between the template and the target. Sadly, your problem is such that the highest location of similarity in this case is incorrect, probably due to noise in your image. – stateMachine Jul 17 '22 at 03:26
  • @stateMachine I agree with you that this is not the ideal scenario for template matching. However, I am required to use openCV template matching in my requirements set. I have transformed the match map to an array of (intensity, and pixel location), and then filtered to keep the top 0.1% of intensities and used K means matching to find the centroids. I have been successful in finding the QR pos squares, but only after invoking openCV QR detect. Upon speaking with my professor, he informed me he was able to use template matching without invoking QR detect and he would prefer I do not use it. – mfergus9 Jul 17 '22 at 03:57
  • Are template ghosts supposed to appear? The bottom right of the first match map seems particularly messed up because we are sliding through a sea of uniform 0 value blacks. I will circle back and try taking only the top 3 pixel intensities instead of the top 0.1% and cut out Kmeans. Thank you for your help! – mfergus9 Jul 17 '22 at 04:01
  • I iterated through all my input set and inspected the top 10 pixel intensities from my match map and they all represented one or two QR positions of the three (mostly one). There's a little dot region of high intensity values if that makes sense, and my method at the moment is to send say the 1000 highest to a Kmeans model with 3 means and I have been successful but only after applying open CV QR Detect to discard all intensities outside the QR code bounded box which is essentially circumventing the purpose of the exercise! – mfergus9 Jul 17 '22 at 04:16
  • to repeat, **template matching does not work here**. give up. did you show your professor your input image? -- that professor can't possibly be a professor if he requires you to use *OpenCV's* `matchTemplate`. that's a very narrow type of template matching. it's merely correlation/convolution. "template matching" generally also implies multi-scale approaches and other notions of "match" that don't go pixel-by-pixel. -- even that type of "template matching" is _not_ used for QR code localization. those patterns _can_ be found with thresholding, contours, and statistics on those contours. – Christoph Rackwitz Jul 17 '22 at 12:08

1 Answers1

1

You use wrong technique for dot detection. Template matching not have size and rotation robustness. The best way to solve your problem is to manually analyze each row of the image and find pattern: N black, N white, 2*N black, N white, N black. This is how real QR code detector works.

For implementation on C++ you can checkout this code. And here you can find some comments about algorithm.

There other way to find this dots. If you use some library for QR detection, this library can provide corners of QR code and you can calculate dot location.

Gralex
  • 4,285
  • 7
  • 26
  • 47