MLKit face detection 's algorithm

Question

MLKit provides good API documentations and guides for Face detection (https://developers.google.com/ml-kit/vision/face-detection). However, I can not find any informations about the algorihms/baseline model or related research papers behind the scene. Can someone provide any suggestions about it's implementation ?

Answer 1

It likely uses blazeface for face detection used on mediapipe. I could not find a direct answer, but when analyzing an apk with mlkit face detection blazeface.tfl can be found on assets folder.

Mediapipe pose detection doc mentions blazepose powers the ML Kit Pose Detection API. So, blazeface is likely to power mlkit on device face detection.

Links to documentation, paper, poster,

https://google.github.io/mediapipe/solutions/face_detection.html

https://arxiv.org/abs/1907.05047

https://docs.google.com/presentation/d/1YCtASfnYyZtH-41QvnW5iZxELFnf0MF-pPWSLGj8yjQ/present?slide=id.g5bc8aeffdd_1_0

https://drive.google.com/file/d/1u6aB6wxDY7X2TmeUUKgFydulNtXkb3pu/view

Answer 2

ML Kit's selection of APIs run on-device or in the cloud. The on-device APIs can work when there's no network connection. The cloud-based APIs is built on top of TensorFlow Lite along with the integrated Neural Network API. So we do not need to integrate AI-based algorithms into mobile apps. I found the explanation on wiki:

Face detection can be regarded as a specific case of object-class detection. Face-detection algorithms focus on the detection of frontal human faces.

A reliable face-detection approach based on the genetic algorithm and the eigen-face technique.

There are other face recognition algorithms, like Principal Component Analysis(PCA) and Linear Discriminate Analysis (LDA) algorithms.

Answer 3

I found ml kit facedetection sdk version has not been update for a long time， and the face mesh sdk is not reliable， maybe mediapipe is the best choice