Android Camera2 Flash Timing Issues

Question

I'm working on an Android camera app that needs to use a fixed (manual) focus, and always uses the flash. I'm having some issues that seem to be with the flash timing. The flash always fires and an image is always acquired, but sometimes the flash doesn't actually illuminate the captured frame. Some frames have flash, some are overexposed, and some are dark; basically it's inconsistent and unpredictable.

I based my code off the Camera2Basic example. I think I've shown all the relevant parts here:

My preview request builder has the following setup

mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_OFF);
float minimumLens = mCameraCharacteristics.get(CameraCharacteristics.LENS_INFO_MINIMUM_FOCUS_DISTANCE);
mPreviewRequestBuilder.set(CaptureRequest.LENS_FOCUS_DISTANCE, minimumLens);
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AWB_MODE,CaptureRequest.CONTROL_AWB_MODE_OFF);
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE,CaptureRequest.CONTROL_AE_MODE_ON_ALWAYS_FLASH);
mPreviewRequestBuilder.set(CaptureRequest.FLASH_MODE, CameraMetadata.FLASH_MODE_OFF);

Then the actual sequence that acquires the pictures (almost straight from Camera2Basic) is:

private void takePicture() {
    runPrecaptureSequence();
}

private CameraCaptureSession.CaptureCallback mCaptureCallback = new CameraCaptureSession.CaptureCallback() {

    private void process(CaptureResult result) {
        switch (mState) {
            case STATE_PREVIEW: {
                break;
            }
            case STATE_WAITING_PRECAPTURE: {
                Integer aeState = result.get(CaptureResult.CONTROL_AE_STATE);
                if (aeState == null ||
                        aeState == CaptureResult.CONTROL_AE_STATE_PRECAPTURE ||
                        aeState == CaptureRequest.CONTROL_AE_STATE_FLASH_REQUIRED) {
                    mState = STATE_CAPTURE;
                }
                break;
            }

            case STATE_CAPTURE: {
                // CONTROL_AE_STATE can be null on some devices
                Integer aeState = result.get(CaptureResult.CONTROL_AE_STATE);
                if (aeState == null || aeState != CaptureResult.CONTROL_AE_STATE_PRECAPTURE) {
                    mState = STATE_PICTURE_TAKEN;
                    captureStillPicture();
                }
                break;
            }
        }
    }

    @Override
    public void onCaptureProgressed(**ARGS**) {
        process(partialResult);
    }

    @Override
    public void onCaptureCompleted(**ARGS**) {
        process(result);
    }

};

private void runPrecaptureSequence() {
    try {          mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER, CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER_START);
        mState = STATE_WAITING_PRECAPTURE;
        mCaptureSession.capture(mPreviewRequestBuilder.build(), mCaptureCallback, mBackgroundHandler);
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

private void captureStillPicture() {
    try {
        final Activity activity = getActivity();
        if (null == activity || null == mCameraDevice) {
            return;
        }

        final CaptureRequest.Builder captureBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
        captureBuilder.addTarget(mImageReader.getSurface());
        captureBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_OFF);           CaptureBuilder.set(CaptureRequest.CONTROL_AWB_MODE,CaptureRequest.CONTROL_AWB_MODE_OFF);
        float minimumLens = mCameraCharacteristics.get(CameraCharacteristics.LENS_INFO_MINIMUM_FOCUS_DISTANCE);
        captureBuilder.set(CaptureRequest.LENS_FOCUS_DISTANCE, minimumLens);
        captureBuilder.set(CaptureRequest.CONTROL_AE_MODE,CaptureRequest.CONTROL_AE_MODE_ON_ALWAYS_FLASH);
        captureBuilder.set(CaptureRequest.FLASH_MODE, CameraMetadata.FLASH_MODE_OFF);
        int rotation = activity.getWindowManager().getDefaultDisplay().getRotation();
        captureBuilder.set(CaptureRequest.JPEG_ORIENTATION, getOrientation(rotation));

        mFileName = getFileNameFromTime() + ".jpg";

        CameraCaptureSession.CaptureCallback CaptureCallback
                = new CameraCaptureSession.CaptureCallback() {

            @Override
            public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                           @NonNull CaptureRequest request,
                                           @NonNull TotalCaptureResult result) {
                resumePreview();
            }
        };

        mCaptureSession.stopRepeating();
        mCaptureSession.capture(captureBuilder.build(), CaptureCallback, null);
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

Saving the image takes place in the ImageReader onImageAvailableListener call and works just fine.

It does seem like the flash is firing before the image is acquired, so I tried the suggestion in this answer, but the FLASH_FIRED conditional suggested never triggered.

Can anybody with better familiarity with Camera2 than me see where I'm screwing up?

Answer 1

i have seen the code of camera2basic https://github.com/googlearchive/android-Camera2Basic and its newer java version https://github.com/android/camera-samples/tree/master/Camera2BasicJava

so basically there are three places in the code where the code sets flash(i am talking about the newer java version)

1) private void createCameraPreviewSession() function

2) private void captureStillPicture()

3) private void unlockFocus()

1) createCameraPreviewSession() function gets called when the preview is about to get displayed (so i you want to start flash from the beginning of the app start call these function accordingly)

2) captureStillPicture() function gets called when it is going to capture an HD picture

3) unlockFocus() function gets called when you have captured image and want to unlock focus when you click an HD picture

so if flash blinks or image gets washed out you may be setting flashmode torch at only one of last two functions set flash mode consitent in last two functions and try to avoid first frame that you capture just after starting flash i was setting flash like this

       int flashFlag=1;
private void setAutoFlash(CaptureRequest.Builder requestBuilder) {
    if (mFlashSupported) {


        if (flashFlag==1)
        {
            requestBuilder.set(CaptureRequest.CONTROL_AE_MODE,CaptureRequest.CONTROL_AE_MODE_ON);
            requestBuilder.set(CaptureRequest.FLASH_MODE,CaptureRequest.FLASH_MODE_TORCH);
        }
        else
        {
            requestBuilder.set(CaptureRequest.CONTROL_AE_MODE,
                CaptureRequest.CONTROL_AE_MODE_ON);
        }


    }
}

i was making flashFlag a global variable and setting it at both the places together and rejecting the first flash HD frame because it usually gets washed out

also do not try to turnOn flash without capturing an HD picture because it will lock you in focus

Answer 2

Add the following three lines to the Camera2Basic Sample:

mPreviewRequestBuilder.set(CaptureRequest.CONTROL_MODE, CaptureRequest.CONTROL_MODE_OFF);
mPreviewRequestBuilder.set(CaptureRequest.SENSOR_SENSITIVITY, 4000);
mPreviewRequestBuilder.set(CaptureRequest.FLASH_MODE, FLASH_MODE_TORCH);

When Flash is activated and a repeatingRequest is started, the manual settings are being overwritten by some kind of default-Settings (Sensor-Sensitivity: 100). But the request clearly states Sensor-Sensitivity should be 4000. I tested it with these two lines in the onCaptureCompleted-method:

Log.d(TAG, "request: "+request.get(CaptureRequest.SENSOR_SENSITIVITY));
Log.d(TAG, "result: "+result.get(CaptureResult.SENSOR_SENSITIVITY));