Reliable program that can detect QR Codes without cv2.QRCodeDetector() or pyzbar library

Question

I am having trouble finding a set of morphological operations that allow me to detect (only) the QR codes in various images using cv2.connectedComponentsWithStats() or cv2.findContours() (but I would prefer to solve this with cv2.connectedComponentsWithStats()).

The images I absolutely need the code to work on are the following:

I have been messing with 2 different codes, one using cv2.connectedComponentsWithStats() and the other cv2.findContours() and some other methods (based off nathancy's answer to Detect a QR code from an image and crop using OpenCV). To test I've been using the following codes:

Using cv2.connectedComponentsWithStats(), the problem with this code is that it captures more than the QR code in the 2nd as you can see bellow. In the 1st it works great and in the 3rd as well if scaled to 0.5, or else it also detects more than the QR code like the 2nd image.

import cv2
import numpy as np

#img = cv2.imread('Code-1.jpg'); scale = 1;
img = cv2.imread('Code-2.jpg'); scale = 1;
#img = cv2.imread('Code-3.jpg'); scale = 0.5;
width = int(img.shape[1] * scale); height = int(img.shape[0] * scale); img = cv2.resize(img, (width, height))

og = img.copy()

gray = np.zeros((img.shape[0], img.shape[1]), dtype=np.uint8)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

gaussianblur = cv2.GaussianBlur(gray, (7,7), 0)

otsuthresh = cv2.threshold(gaussianblur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

edges = cv2.Canny(otsuthresh, threshold1=100, threshold2=200)

dilate = cv2.dilate(edges,(5,5),iterations=1)

num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(dilate, 8, cv2.CV_32S)

for i in range(1,num_labels):
    objint = (labels == i).astype(np.uint8)*255/i
    x = stats[i, cv2.CC_STAT_LEFT]
    y = stats[i, cv2.CC_STAT_TOP]
    w = stats[i, cv2.CC_STAT_WIDTH]
    h = stats[i, cv2.CC_STAT_HEIGHT]
    area = stats[i, cv2.CC_STAT_AREA]
    ratio = w / float(h)
    (cX, cY) = centroids[i]
    if area > 500 and (ratio > .95 and ratio < 1.05) and (w < 0.99*img.shape[1]):
        cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
        ROI = og[y:y + h, x:x + w]
        cv2.imwrite('ROI.png', ROI)

cv2.imshow('image', img)
cv2.imshow('QR code', ROI)

Using cv2.findContours(), this one can't detect any of the QR codes in the images in which the code must not fail, but can detect in some other random images

import cv2
import numpy as np

#img = cv2.imread('Code-1.jpg'); scale = 1;
img = cv2.imread('Code-2.jpg'); scale = 1;
#img = cv2.imread('Code-3.jpg'); scale = 0.5;
width = int(img.shape[1] * scale); height = int(img.shape[0] * scale); img = cv2.resize(img, (width, height))

og = img.copy()

gray = np.zeros((img.shape[0], img.shape[1]), dtype=np.uint8)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

gaussianblur = cv2.GaussianBlur(gray, (7,7), 0)

otsuthresh = cv2.threshold(gaussianblur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))
closed = cv2.morphologyEx(otsuthresh, cv2.MORPH_CLOSE, kernel, iterations=3)

contours = cv2.findContours(closed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

if len(contours) == 2:
    contours = contours[0]
else:
    contours = contours[1]

for cnt in contours:
    perim = cv2.arcLength(cnt, True)
    approx = cv2.approxPolyDP(cnt, 0.05 * perim, True)
    x,y,w,h = cv2.boundingRect(approx)
    area = cv2.contourArea(cnt)
    ratio = w / float(h)
    if len(approx) == 4 and area > 1000 and (ratio > .80 and ratio < 1.2):
        cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 4)
        ROI = og[y:y + h, x:x + w]
        cv2.imwrite('ROI.png', ROI)

cv2.imshow('image', img)
cv2.imshow('QR code', ROI)

Thank you for reading and if I wasn't clear on something please let me know.

Filipe Almeida

detection of QR codes has **nothing** to do with CC or findContours. drop those functions. review how the qr code detection is supposed to be used. there are examples everywhere. — Christoph Rackwitz, Nov 24 '21 at 20:49
thank you for your input, but It is simply the way it was requested to me — Filipe Almeida, Nov 24 '21 at 20:56
Detecting QR codes this way in complex images is challenging. They are essentially characterized by dense color inversions, hence generate much "gradient activity". But so can do other features (such as populated PCBs). More distinctive features are constant cell size and cell alignment on a grid. Unfortunately, as the cells touch randomly, this is pretty difficult to assess. If the images are easy to binarize, you may try your luck by detecting complex, wiggly outlines, with a high edge length/area ratio. — , Nov 24 '21 at 21:20
If someone asks a heart surgeon to do open-heart surgery with a butter knife and a spatula, the surgeon will say “that cannot be done”. Likewise, if someone asks you for this task with these constraints, you need to explain to them that the constraints are inappropriate, and what a good solution to the problem would look like. — Cris Luengo, Nov 24 '21 at 21:36
I appreciate your comments and I do now realize this is not the way to approach the task of detecting QR codes by a long shot. Still, I need to try and accomplish it this way. If it worked for 2 of the 3 images it might work for the other one — Filipe Almeida, Nov 24 '21 at 22:03
first find the concentric squares that are the defining feature of QR codes. those aren't arbitrary. those are required to localize it. findContours will do the job. retrieve hierarchy too. filter the contours. **most** QR detection finds those first. I don't know any other approach because the rest of the bits are quasi-random or just timing tracks, so not useful for detecting/localizing. -- if they tell you to use findContours, you shouldn't just run at the problem, trying random things. that's wrong. READ about what other people have done. that's called "literature review". — Christoph Rackwitz, Nov 25 '21 at 01:18
@CrisLuengo: detecting QR codes without using the finder patterns is not a silly idea. There are cases where these patterns are damaged or hidden. In addition, the same technique will apply to other important 2D codes, in particular Data Matrix. It is in fact used by high-end readers. — , Nov 25 '21 at 07:33
"If it worked for 2 of the 3 images it might work for the other one": how optimistic ! Though these images are clean and very well contrasted, they significantly differ in cell size, and this is not a minor difference. — , Nov 25 '21 at 07:35

score 0 · Answer 1 · answered Dec 05 '22 at 22:55

Maybe, you could try QReader. It is just a wrapper of OpenCV, Pyzbar and other QR detection and image filtering methods, but it works quite out-of-the-box for those cases.

from qreader import QReader
from matplotlib import pyplot as plt
import cv2

if __name__ == '__main__':
    # Initialize QReader
    detector = QReader()
    for img_path in ('0oOAF.jpg', 'HXlS8.jpg', '5fFTo.jpg'):
        # Read the image
        img = cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)

        # Detect the QR bbox
        found, bbox = detector.detect(image=img)
        if found:
            # Draw the bbox
            x1, y1, x2, y2 = bbox
            cv2.rectangle(img=img, pt1=(x1, y1), pt2=(x2, y2), color=(0, 255, 0), thickness=2)
            # Save the image
            plt.imshow(img)
            plt.savefig(f"{img_path}-bbox.png")

That's the output it gives:

Reliable program that can detect QR Codes without cv2.QRCodeDetector() or pyzbar library

1 Answers1