DLIB's Upper and lower eye lid detection accuracy is not correct for closed eye

Question

We are using DLIB for face detection, which is extended to detect whether eye is open or closed.

It works really well but eye is closed completely for some images it is not giving proper points.

e.g. Proper Image

As you can see, RED points are properly placed.

Incorrect detection as below image

As you can see the point's plotted in blue are not in correct position. Ideally distance between upper and lower eye-lid should be Zero.

But this is not the case. We searched if any tuning can be made but no luck.

Alternative solution can be, apply some filter after dlib processing to detect Iris etc. but looking for any better solution for this.

score 1 · Answer 1 · answered Sep 12 '19 at 22:42

There are few improvements you could do. First you need to use gray frame for better tracking:

# Load frames from the camera
while True:
    _, frame = cap.read()
    # Use gray frame for better tracking
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

The second improvement you could do is to create a new facial landmark (top center of the eye and bottom center of the eye), you do so by using simpel math:

# Create new facial landmarks
def midpoint(p1, p2):
    return int((p1.x + p2.x)/2), int((p1.y + p2.y)/2)

Then you should have points that are located on top and bottom center of each eye. You might ask: well how can that help? Well, you could use these points and draw a line between them like so:

# Draw line between different facial landmarks
ver_line = cv2.line(frame, center_top, center_bottom, (0, 255, 0), 2)

Then do the same thing with a horizontal line:

# Draw line between different facial landmarks
hor_line = cv2.line(frame, left_point, right_point, (0, 255, 0), 2)

Then calculate the length of the horizontal and vertical line, and take the ratio of it:

# Calculating length of the lines
hor_line_lenght = hypot(
    (left_point[0] - right_point[0]), (left_point[1] - right_point[1]))
ver_line_lenght = hypot(
    (center_top[0] - center_bottom[0]), (center_top[1] - center_bottom[1]))
ratio = hor_line_lenght/ver_line_lenght
return ratio

Thank you for reply and suggestions. We tried with gray scale but same output from Dlib. Regarding ratio, this will helpful when dlib detects proper points, in case of eye is closed. — Banng, Sep 13 '19 at 04:37

DLIB's Upper and lower eye lid detection accuracy is not correct for closed eye

1 Answers1