CaptureResult passed to DngSaver in ImageSaver is null camera2 api

Question

In my quest to build a camera app using camera2api I have a small issue while trying to save a raw image.

I assign the capture result to a member in the following bit of code.

@Override

public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {
                        super.onCaptureCompleted(session, request, result);


                        mCaptureResult = result;
                        Toast.makeText(getApplicationContext(),
                                "Image Captured",Toast.LENGTH_SHORT).show();

                    }
                };

When I debug it at this location the member is assigned and not null. However when It throws a null error at the DngCreator in ImageSaver class.

My Listeners:

private ImageReader mImageReader;
private ImageReader.OnImageAvailableListener mOnImageAvailableListener =
        new ImageReader.OnImageAvailableListener() {
            @Override
            public void onImageAvailable(ImageReader imageReader) {
                mBackgroundHandler.post(new ImageSaver(imageReader.acquireNextImage(),mUiHandler,
                        mCaptureResult, mCameraCharacteristics));
            }
        };

private ImageReader mRawImageReader;
private ImageReader.OnImageAvailableListener mOnRawImageAvailableListener =
        new ImageReader.OnImageAvailableListener() {
            @Override
            public void onImageAvailable(ImageReader imageReader) {
                mBackgroundHandler.post(new ImageSaver(imageReader.acquireNextImage(),mUiHandler,
                        mCaptureResult, mCameraCharacteristics));
            }
        };

My Image Saver class:

    private ImageSaver(Image image, Handler handler, CaptureResult captureResult,
                       CameraCharacteristics cameraCharacteristics) {
        mImage = image;
        mHandler =handler;
        mCaptureResult = captureResult;
        mCameraCharacteristics = cameraCharacteristics;
    }



    @Override
    public void run() {
        int format = mImage.getFormat();
        switch(format){
            case ImageFormat.JPEG:
                ByteBuffer byteBuffer = mImage.getPlanes()[0].getBuffer();
                byte[] bytes = new byte[byteBuffer.remaining()];
                byteBuffer.get(bytes);

                FileOutputStream fileOutputStream = null;
                try {
                    fileOutputStream = new FileOutputStream(mImageFile);
                    fileOutputStream.write(bytes);
                }
                catch (IOException e){
                    e.printStackTrace();
                }
                finally {
                    mImage.close();
                    if(fileOutputStream != null){
                        try{
                            fileOutputStream.close();
                        }
                        catch (IOException e){
                            e.printStackTrace();
                        }
                    }
                    //Message message = mHandler.obtainMessage();
                    //message.sendToTarget();
                }
                break;
            case ImageFormat.RAW_SENSOR:
                DngCreator dngCreator = new DngCreator(mCameraCharacteristics,mCaptureResult);
                FileOutputStream rawFileOutputStream = null;
                try {
                    rawFileOutputStream = new FileOutputStream(mRawImageFile);
                    dngCreator.writeImage(rawFileOutputStream, mImage);
                }
                catch (IOException e){
                    e.printStackTrace();
                }
                finally{
                    mImage.close();
                    if(rawFileOutputStream != null){
                        try {
                            rawFileOutputStream.close();
                        }
                        catch (IOException e){
                            e.printStackTrace();
                        }
                    }
                }
                break;
        }


    }
}

Right now it gives me an error when I try to initialize the DngCreator and the stacktrace is as follows.

FATAL EXCEPTION: Camera2 Background Thread Process: com.something.something, PID: 5162 java.lang.IllegalArgumentException: Null argument to DngCreator constructor at android.hardware.camera2.DngCreator.<init>(DngCreator.java:89) at com.something.something.ControlCameraActivity$ImageSaver.run(ControlCameraActivity.java:328) at android.os.Handler.handleCallback(Handler.java:739) at android.os.Handler.dispatchMessage(Handler.java:95) at android.os.Looper.loop(Looper.java:148) at android.os.HandlerThread.run(HandlerThread.java:61)

Can someone please help me fix this? IF you need any more information from me do let me now. Thanks

Answer 1

There's no guarantee that onCaptureCompleted() happens before OnImageAvailable(). In fact, since RAW image buffers are often ready earlier than the image processing for preview and result metadata completes, it's very likely that onCaptureCompleted() will happen after onImageAvailable().

If onImageAvailable happens first, you don't have a capture result yet.

Instead, you should wait for both the image and the capture result to be done, then create the ImageSaver. As one option, have both callbacks save their respective outputs (capture result and Image) in some shared location, and then check if both are non-null - if so, invoke the ImageSaver. Then it doesn't matter which runs first, the one that runs second launches the ImageSaver.

Answer 2

Firstly, when you call imageReader.acquireNextImage() set it in a Image object and after use it do a Image.close. ImageReaders are really dangerous with garbage collector and also if you have a low number in your ImageReader maxImages, you will fill it pretty fast because you are not closing the images.

Second, about your references, where is your savingImage method? image is not a parsable object, so, if you are sending it to a service or another class maybe you are having problems with that.

Finally, as @Eddy say, sometimes onCaptureComplete is not called before your OnImageAvailable. So, you have 2 solutions about this:

Just add a check for avoid FC in your apps:

if (mImage != null && mCameraCharacteristics != null && mDngResult != null) { 
//process your dng
}

Or, you can create a a Listener. So, when you recieve your onImageAvailable(), you can put a petition in a Custom List, or Hastable.

There, check if the List have any captureResult available, if you have a CaptureResult, take it and process your DNG, if not instantiate a Listener which will be called in your onCaptureComplete. There you just need to check if the listener is null, and if it's not, make the call to "process" the image now that your result is available.

public class MyResultList {

    private DngListener mDngListener;
    private Hashtable<String, CaptureResult> mDngCaptureResults;

    public int getSize() {
        return mDngCaptureResults.size();
    }

    public Object getResult() {
        if (mDngCaptureResults.size() == 0){
            mDngListener = new DngListener() {
                @Override
                public void onResultAvailable() {
                    //Now result is not null, so you can get the available item
                    YourSavingImage(mDngCaptureResults.get(0));
                }
            }
            return null;
        } else {
            return mDngCaptureResults.get(0);
        }
    }

    public void add(CaptureResult result, String index){
        mDngCaptureResults.put(index,result);
    }
}

I just made that idea, modify it for your objectives. It's quite simple and maybe doesn't work so good if you have a lot of dependencies.