Python cvzone - error when making AR paint

Question

I am trying to make an AR paint program using python and opencv library, but for some reason when I run my code the program does not draw the line and I don't know why. I followed this tutorial on yt: https://www.youtube.com/watch?v=ZiwZaAVbXQo&t=201s and simpyfied it by cutting out the color selection menue and removing the HandTrackingModule.py file because I just imported the HandDetector from cvzone.HandTrackingModule library and edited some parts because I use the latest version of opencv.

So the error is that my program does not draw the line.

I am using cvzone version 1.5.2 (the latest)

This is my code:

import cv2
import numpy as np
import os
from cvzone.HandTrackingModule import HandDetector #----> this is the library I imported whcih was made by him
 
brushThickness = 25

drawColor = (0, 0, 0)
 
cap = cv2.VideoCapture(0)
cap.set(3, 640)
cap.set(4, 480)
 
detector = HandDetector(detectionCon=0.65,maxHands=1) #here I imported the HandDetector
xp, yp = 0, 0
imgCanvas = np.zeros((480, 640, 3), np.uint8)
 
while True:
 
    # 1. Import image
    success, img = cap.read()
    img = cv2.flip(img, 1)
 
    # 2. Find Hand Landmarks
    hands, img = detector.findHands(img, flipType=False)
    
    if hands:
        lmList = hands[0]['lmList'] #this was the change I made because my version does not support his line
        #he used : lmList = detector.findPosition(img, draw=False)
 
        if len(lmList) != 0:
    
            # tip of index and middle fingers
            x1, y1 = lmList[8][:2]
            x2, y2 = lmList[12][:2]
    
            # 3. Check which fingers are up
            fingers = detector.fingersUp(hands[0])

            # 4. If two index fingers are up - drawing mode -----> somewhere here is the error becuase my program does not draw the line
            if fingers[1] == 1 and fingers[2] == 1:
                cv2.circle(img, (x1, y1), 15, drawColor, cv2.FILLED)

                if xp == 0 and yp == 0:
                    xp, yp = x1, y1
    
                cv2.line(img, (xp, yp), (x1, y1), drawColor, brushThickness)
    
                xp, yp = x1, y1
    
            # Clear Canvas when all fingers are up
            if all (x >= 1 for x in fingers):
                imgCanvas = np.zeros((480, 640, 3), np.uint8)
    
        imgGray = cv2.cvtColor(imgCanvas, cv2.COLOR_BGR2GRAY)
        _, imgInv = cv2.threshold(imgGray, 50, 255, cv2.THRESH_BINARY_INV)
        imgInv = cv2.cvtColor(imgInv,cv2.COLOR_GRAY2BGR)
        img = cv2.bitwise_and(img, img, imgInv)
        img = cv2.bitwise_or(img, img, imgCanvas)

    cv2.imshow("Image", img)
    cv2.waitKey(1)

you should run your program in a debugger and examine its behavior. if it doesn't draw, program flow didn't reach those drawing calls. can you run your program and watch it execute and see why it's not reaching those draw calls? — Christoph Rackwitz, Oct 25 '21 at 15:20
how is it supposed to draw a *visible* line when the first and second point are identical? what concrete argument values are passed to the `line` call, for instance? — Christoph Rackwitz, Oct 25 '21 at 15:24
I ran it in debugger and the code doesn't stop anywhere, I think I just made a mistake in my code — gasper_thebeast101, Oct 25 '21 at 15:28
They are identical, but because of the loop the xp and yp are previous x1 and y1 position - because they are updated after the line was called, I just followed the yt tutorial and simmilar code worked for him but not for me. — gasper_thebeast101, Oct 25 '21 at 15:29
great, then look at the values in the variables, and the values you pass into the function. look again in the next iteration. don't just stop when the program works correctly. keep looking. — Christoph Rackwitz, Oct 25 '21 at 15:29
I tryied many times but I just don't find the error, I think I made a mistake when drawing the line or after that when merging canvas and img — gasper_thebeast101, Oct 25 '21 at 15:32

score 0 · Accepted Answer · answered Jun 11 '22 at 08:05

I messed up the loop orders, Here is the solution (I also added that the line erases after all fingers are raised):

import cv2
import numpy as np
from cvzone.HandTrackingModule import HandDetector


brushThickness = 25

drawColor = (255, 0, 255)
 
cap = cv2.VideoCapture(0)
cap.set(3, 640)
cap.set(4, 480)
 
detector = HandDetector(detectionCon=0.65,maxHands=1)
xp, yp = 0, 0
imgCanvas = np.zeros((480, 640, 3), np.uint8)

 
while True:
 
    success, img = cap.read()
    img = cv2.flip(img, 1)
 
    hands, img = detector.findHands(img)
    
    if hands:
        lmList = hands[0]['lmList']
        if len(lmList) != 0:
            
            # tip of index and middle fingers
            x1, y1 = lmList[8][:2]
            x2, y2 = lmList[12][:2]
    
            #Check which fingers are up
            fingers = detector.fingersUp(hands[0])
            
            #Drawing Mode - Index finger is up
            if fingers == [0, 1, 0, 0, 0]:
                cv2.circle(img, (x1, y1), 15, drawColor, cv2.FILLED)
                #print("Drawing Mode")
                if xp == 0 and yp == 0:
                    xp, yp = x1, y1

                #ce je sam img pol update vsakic in nemors risat, rise v canvas 
                cv2.line(imgCanvas, (xp, yp), (x1, y1), drawColor, brushThickness)
              
                xp, yp = x1, y1

            #reset when two fingers are up
            if fingers == [0, 1, 1, 0, 0]:
                xp, yp = x1, y1
                
                
            # Clear Canvas when all fingers are up
            if all (x >= 1 for x in fingers):
                 imgCanvas = np.zeros((480, 640, 3), np.uint8)
    
    #zdruzis canvas pa img 
    imgGray = cv2.cvtColor(imgCanvas, cv2.COLOR_BGR2GRAY)
    _, imgInv = cv2.threshold(imgGray, 50, 255, cv2.THRESH_BINARY_INV)
    imgInv = cv2.cvtColor(imgInv,cv2.COLOR_GRAY2BGR)
    img = cv2.bitwise_and(img,imgInv)
    img = cv2.bitwise_or(img,imgCanvas)
 

    cv2.imshow("Image", img)
    #cv2.imshow("Canvas", imgCanvas)
    cv2.waitKey(1)

Python cvzone - error when making AR paint

1 Answers1