Firebase MLKit Facedetection Slow For Android

Question

I want to detect face from gallery Image (Bitmap).

Issues

1. I noticed that Firebase MLKIT is performing very slow on Gallery Image Bitmap.
2. Can i still used mobile vision api for detection of face in image.( I want only detect face, dont want eyes,nose,etc)
3. What should i do to improve performance for detecting face with Firebase MLKIT.
4. I used Firebase Image Labeling. Firebase Image Labeling is performed fast but Face Detection is very slow comparatively.

I tried with Mobile vision Api and detected face successfully. On website of mobile vision api, they have mentioned about Firebase MLKIT. I also tried firebase ML Kit and detected face successfully. I followed this link for demo: [https://github.com/hitanshu-dhawan/FirebaseMLKit]

Library Versions:

implementation 'com.google.firebase:firebase-core:17.0.1'
implementation 'com.google.firebase:firebase-ml-vision:22.0.0'
implementation 'com.google.firebase:firebase-ml-vision-face-model:18.0.0'
implementation 'com.google.firebase:firebase-ml-vision-image-label-model:18.0.0' 

---

    FirebaseVisionFaceDetectorOptions option =
     new FirebaseVisionFaceDetectorOptions.Builder()
    .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE)
    .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS)
    .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS)
    .build();

    FirebaseVisionFaceDetector detector = FirebaseVision.getInstance()
                .getVisionFaceDetector(option);

        detector.detectInImage(image).addOnSuccessListener(
                new OnSuccessListener<List<FirebaseVisionFace>>() {
                    @Override
                    public void onSuccess(List<FirebaseVisionFace> faces) {
    }

Is am i doing something wrong?

Answer 1

I think you can change .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE) To .setPerformanceMode(FirebaseVisionFaceDetectorOptions.FAST)

May be it will improve speed of detecting

Answer 2

I have the same issue but with text recognition, very slow on Android, around 1 image per second (Huawei Y6 2018) but incredibly fast on iOS, I can easily run 10 frames per second on iPhone 6s. I have tried all image formats which Firebase support, during my tests fastest varian is ByteBuffer, so I convert Bitmap to ByteBuffer before start recognision.

Answer 3

Firebase face detector is very slow if you instantiate your FirebaseVisionImage directly from a bitmap, as in:

FirebaseVisionImage visionImage = FirebaseVisionImage.fromBitmap(bitmap);

A solution is to convert the bitmap into a byte array (byte[]) and use this other constructor to create the FirebaseVisionImage:

FirebaseVisionImage visionImage = FirebaseVisionImage.fromByteArray(byteArray, metadata);

This fact does not seem to be documented. I found it in a comment on this GitHub issue and used the proposed technique, reducing face detection time 6 times aprox. Also in that link there's a code snippet from GitHub user jllarraz to transform the bitmap to an nv21 byte array.

Answer 4

Thanks for above solution but finally i am able to use face detector at very fast. I just calculate insamplesize of the bitmap and reduce the size of bitmap. As bitmap size is very less, able to process face detection at much faster and then once i got the coordinate, again map that coordinate with original image by multiplying insamplesize. This way i achieved image processing with FirebaseVisionFaceDetectorOptions at fast speed. Below is code for calculating insamplesize.

public static int calculateInSampleSize(
            BitmapFactory.Options options, int reqWidth, int reqHeight) {
    // Raw height and width of image
    final int height = options.outHeight;
    final int width = options.outWidth;
    int inSampleSize = 1;

    if (height > reqHeight || width > reqWidth) {

        final int halfHeight = height / 2;
        final int halfWidth = width / 2;

        // Calculate the largest inSampleSize value that is a power of 2 and keeps both
        // height and width larger than the requested height and width.
        while ((halfHeight / inSampleSize) >= reqHeight
                && (halfWidth / inSampleSize) >= reqWidth) {
            inSampleSize *= 2;
        }
    }

    return inSampleSize;
}