OpenCV: Lucas Kanade applied within certain area (to detect facial features)

Question

I am trying to preform face tracking with the Lucas Kanade algorithm with Haar Cascade Classification. The Lucas Kanade is successful and can track the user, but unfortunately, some of the good features to detect points are wasted on corners in the background. I wish to use Haar Cascade's ability to detect the fact to get coordinates of detected face and apply Lucas Kanade to only within that restricted area.

Basically, I want to use Haar Cascade to detect fact, get x, y, w, and h values, and use those coordinates to apply Lucas Kanade within that restricted area (so that none are wasted on assigning good features to the background and only facial features are detected)

The line of code that is doing the Lucas Kanade algorithm is this code:

p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)

How do I do that?

Code:

from matplotlib import pyplot as plt
import numpy as np

import cv2

rectangle_x = 0

face_classifier = cv2.CascadeClassifier('haarcascades/haarcascade_frontalface_default.xml')

cap = cv2.VideoCapture(0)


# params for ShiTomasi corner detection
feature_params = dict( maxCorners = 200,
                       qualityLevel = 0.01,
                       minDistance = 10,
                       blockSize = 7 )

# Parameters for lucas kanade optical flow
lk_params = dict( winSize  = (15,15),
                  maxLevel = 2,
                  criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

# Create some random colors
color = np.random.randint(0,255,(100,3))

# Take first frame and find corners in it
ret, old_frame = cap.read()

cv2.imshow('Old_Frame', old_frame)
cv2.waitKey(0)
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)
restart = True
face = face_classifier.detectMultiScale(old_gray, 1.2, 4)

if len(face) == 0:
    print "This is empty"

for (x,y,w,h) in face:
    focused_face = old_frame[y: y+h, x: x+w]



cv2.imshow('Old_Frame', old_frame)

face_gray = cv2.cvtColor(old_frame,cv2.COLOR_BGR2GRAY)

gray = cv2.cvtColor(focused_face,cv2.COLOR_BGR2GRAY)

corners_t = cv2.goodFeaturesToTrack(gray, mask = None, **feature_params)
corners = np.int0(corners_t)




for i in corners:
    ix,iy = i.ravel()
    cv2.circle(focused_face,(ix,iy),3,255,-1)
    cv2.circle(old_frame,(x+ix,y+iy),3,255,-1)

    print ix, " ", iy

plt.imshow(old_frame),plt.show()


##########

#############################
p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)
#############################
# Create a mask image for drawing purposes
mask = np.zeros_like(old_frame)

print "X: ", x
print "Y: ", y

while(1):
    ret,frame = cap.read()
    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # calculate optical flow
    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)

    # Select good points
    good_new = p1[st==1]
    good_old = p0[st==1]

    # draw the circles
    for i,(new,old) in enumerate(zip(good_new,good_old)):
        a,b = new.ravel()
        c,d = old.ravel()
        cv2.circle(frame,(a, b),5,color[i].tolist(),-1)
        if i == 99:
            break

    cv2.imshow('frame',frame)
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break

    # Now update the previous frame and previous points
    old_gray = frame_gray.copy()
    p0 = good_new.reshape(-1,1,2)

cv2.destroyAllWindows()
cap.release()

if the aim is to get facial features - why not use a library like flandmark - http://cmp.felk.cvut.cz/~uricamic/flandmark/index.php ? Any reason to prefer Haar+LK ? — kiranpradeep, Jan 15 '15 at 06:02
One reason is I am programming it in Python, and I don't know if it's even capatable. And it is also because I am doing this as a learning experience, I don't want a library to do it for me. — user3377126, Jan 15 '15 at 06:58
instead of `mask = None` just use a mask in the region where you detected the face... e.g. mask = draw a white rectangle for each face you detected on a black image of same size as input image (single channel). — Micka, Jan 15 '15 at 13:38
What would be the syntax for that? I am having trouble assigning the mask attribute with my detected face coordinates (I keep getting errors) — user3377126, Jan 15 '15 at 15:52

Abhijeet Sinha · Answer 1 · 2016-09-27T06:33:50.027

1

Here is the code snippet:

p0 = np.array([[[x,y]], [[x0,y0]]], np.float32)

just replace p0 in original code and and assign x,x0... with your desired points - make sure its a 2d array - and the type is float 32 for single precision

edited Sep 27 '16 at 06:33

answered Sep 25 '16 at 20:58

Abhijeet Sinha

71
1
6

I'm not sure how this solves the problem. Could you explain? – James K Sep 25 '16 at 21:35
In tutorials cv2.goodFeaturesToTrack is used to assign p0 (old frame) the corners instead the line mentioned above assigns p0 with user desired specific points (like x , y centroid coordinates of face detection) so now when ever p1 (new point ) is calculated it taken account p0 which is pointing x and y. So finally the process remains same just we are pointing towards desired location instead. I have tried this code and it works fine! – Abhijeet Sinha Sep 27 '16 at 06:31
instead of [200,200] just add user point of interest as [x,y] that's all as in p0 = np.array([[[x,y]], [[x0,x1]]], np.float32) – Abhijeet Sinha Sep 27 '16 at 06:32

OpenCV: Lucas Kanade applied within certain area (to detect facial features)

1 Answers1