1

I am using MLKit for face detection on android. I am following the official documentation which can be found here - https://developers.google.com/ml-kit/vision/face-detection/android. I am successfully able to detect faces using the rear camera, but no faces are detected when I switch to the front camera. According to what I have understood from this forum, the problem might be rotation compensation. I have used the same code for this purpose as shown in the documentation -

private static final SparseIntArray ORIENTATIONS = new SparseIntArray();

static {
    ORIENTATIONS.append(Surface.ROTATION_0, 90);
    ORIENTATIONS.append(Surface.ROTATION_90, 0);
    ORIENTATIONS.append(Surface.ROTATION_180, 270);
    ORIENTATIONS.append(Surface.ROTATION_270, 180);
}

/**
 * Get the angle by which an image must be rotated given the device's current
 * orientation.
 */
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
private int getRotationCompensation(String cameraId, Activity activity, Context context)
        throws CameraAccessException {
    // Get the device's current rotation relative to its "native" orientation.
    // Then, from the ORIENTATIONS table, look up the angle the image must be
    // rotated to compensate for the device's rotation.
    int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
    int rotationCompensation = ORIENTATIONS.get(deviceRotation);

    // On most devices, the sensor orientation is 90 degrees, but for some
    // devices it is 270 degrees. For devices with a sensor orientation of
    // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
    CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
    int sensorOrientation = cameraManager
            .getCameraCharacteristics(cameraId)
            .get(CameraCharacteristics.SENSOR_ORIENTATION);
    rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;

    return rotationCompensation;
}

For changing the camera, I just change a global variable cameraID and no other changes are made in the code.

EDIT- In case of front camera, generating input image with rotation = getRotationCompensation(...) - 180 and not subtracting 180 in case of rear camera worked for me. Not sure why that works though. I would still appreciate any explanation regarding this or how this can be avoided.

Achint Agrawal
  • 23
  • 6
  • Are you using CameraX or Camera2? – Hoi Jul 24 '20 at 17:58
  • I'm using Camera2 – Achint Agrawal Jul 24 '20 at 21:23
  • It may be because the front camera provides you the rotated bitmap which is rotated over 180 degree hence when calculating the logic for face detection it does not worked. You must need to rotate the image to -180 degree in case where you get the output. Hence your code worked when -180 is applied to actual rotation. – Khush Parmar Sep 26 '22 at 15:33

1 Answers1

2

The calculation in the above code snippet may not work well for front-facing camera since the image flip.

Could you try to define:

static {
  ORIENTATIONS.append(Surface.ROTATION_0, 0);
  ORIENTATIONS.append(Surface.ROTATION_90, 90);
  ORIENTATIONS.append(Surface.ROTATION_180, 180);
  ORIENTATIONS.append(Surface.ROTATION_270, 270);
}

And use

if (lensFacing == CameraSelector.LENS_FACING_FRONT) {
  rotationCompensation = (sensorOrientation + rotationDegrees) % 360;
} else { // back-facing
  rotationCompensation = (sensorOrientation - rotationDegrees + 360) % 360;
}

to replace the

rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;

We will also update the code snippet in the dev doc to reflect that.

Shiyu
  • 875
  • 4
  • 5
  • thank you for your reply that worked for the rear camera but not the front. Subtracting 180 after making these changes still does the trick though – Achint Agrawal Jul 24 '20 at 21:42
  • Double check you redefined ORIENTATIONS and replaced the rotation computation. Could you let me know your device type if it still does not work? – Shiyu Jul 25 '20 at 03:33
  • I cross checked, the ORIENTATIONS, it still doesn't work. Just to make sure, by rotationDegrees in your answer, you meant rotationCompensation right? There is no other variable in the code named rotationDegrees. I am using Redmi Note 5 for testing.(Android 8) – Achint Agrawal Jul 25 '20 at 08:18
  • 1
    Yes, it is `degreeCompensation`. Could you provide the `deviceRotation`, `sensorOrientation`, `ORIENTATIONS.get(deviceRotation)` and the final `degreeCompensation` value? and when it is portrait mode or landscape mode? Thanks! – Shiyu Jul 27 '20 at 03:45