Unable to detect facial landmarks using OpenCV2

Question

I have developed a script using dlib and cv2 to draw facial landmarks on images having one face in that image. Here is the scripts;

import cv2
import dlib

img_path = 'landmarks.png'
detector = dlib.get_frontal_face_detector()

shape_predictor = 'shape_predictor_68_face_landmarks.dat'
predictor = dlib.shape_predictor(shape_predictor)


count = 1
ready = True
while ready:
    frame = cv2.imread("demo.jpg")
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    faces = detector(gray)
    for face in faces:
        x1 = face.left()
        y1 = face.top()
        x2 = face.right()
        y2 = face.bottom()
        cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 3)

        landmarks = predictor(gray, face)

        for n in range(0, 68):
            x = landmarks.part(n).x
            y = landmarks.part(n).y
            cv2.circle(frame, (x, y), 4, (255, 0, 0), -1)

    cv2.imshow("Frame", frame)
    cv2.waitKey(0)
    ready = False

Now, here what makes me crazy. When I try to download any of the images(with or without mask) from google to test it, this script is working fine. Likewise, you can see these results such as,

But when I try over these following images, it does nothing.

I have made a couple of searches over the internet but I haven't found anything that is serving the current purpose.

Even, I have tried the combination of

• cv2.CascadeClassifier("haarcascade_frontalface_default.xml")
• eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
• m_cascade = cv2.CascadeClassifier('haarcascade_mcs_mouth.xml')

I also have looked into the following useful links out there;

• Face Bounding Box

• Detect Face Landmarks in Android (Even not same domain)

• Landmarks detection

• OpenCV2 Detect Facial Landmarks

but it's also not working on these images. CV2 detector shows an empty list when I debug through script such as;

I just want to draw fiducial landmarks using the above images. What would the best possible solution, I can go through? Maybe, I am missing something in cv2 & Dlib, but unable to get the results as required.

I have also find the confidence score for dlib using the recommended implementation from a Stack Overflow geek such as;

import dlib

detector = dlib.get_frontal_face_detector()

img = dlib.load_rgb_image('demo.jpg')
dets, scores, idx = detector.run(img, 1, -1)
for i, d in enumerate(dets):
    print("Detection {}, score: {}, face_type:{}".format(
        d, scores[i], idx[i]))

Here is the result of a confidence score for the first image in the above-given images in the second row;

Looking forward to getting better research from any of the awesome guys out there. Thanks

Answer 1

First, I might try to see if you can get confidence scores out of dlib. I'm not sure what the confidence threshold is, but perhaps faces are detected that are below the limit. From the dlib Git Repo, here is an example of how to get confidence from the detections:

if (len(sys.argv[1:]) > 0):
    img = dlib.load_rgb_image(sys.argv[1])
    dets, scores, idx = detector.run(img, 1, -1)
    for i, d in enumerate(dets):
        print("Detection {}, score: {}, face_type:{}".format(
            d, scores[i], idx[i]))

Alternatively, consider another face detector, for example a CNN-based detector like this MobileNet SSD face detector. I have not used this particular model, but I have used similar models, like the Google TPU-based face detector model here with very good results.

Answer 2

Download "shape_predictor_68_face_landmarks.dat" link: enter link description here

100% working Code Try This One:

import cv2
import dlib
import numpy as np

img= cv2.imread('Capture 8.PNG')
gray=cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

p = "shape_predictor_68_face_landmarks.dat"
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(p)
faces = detector(gray)

for face in faces:
  x1=face.left()
  y1=face.top()
  x2=face.right()
  y2=face.bottom()
  cv2.rectangle(img, (x1,y1), (x2,y2),(0,255,0),3)
  landmarks=predictor(gray, face)
  for n in range(0,68):
    x=landmarks.part(n).x
    y=landmarks.part(n).y
    cv2.circle(img, (x, y), 4, (0, 0, 255), -1)

cv2.imshow(img)