How to save a YUV_420_888 image?

Question

I built my own camera app with the camera2 API. I started with the sample "camera2Raw" and I added YUV_420_888 support instead of JPEG. But now I am wondering how I save the images in the ImageSaver!?

Here is my code of the run method:

@Override
    public void run() {
        boolean success = false;
        int format = mImage.getFormat();
        switch(format) {
            case ImageFormat.RAW_SENSOR:{
                DngCreator dngCreator = new DngCreator(mCharacteristics, mCaptureResult);
                FileOutputStream output = null;
                try {
                    output = new FileOutputStream(mFile);
                    dngCreator.writeImage(output, mImage);
                    success = true;
                } catch (IOException e) {
                    e.printStackTrace();
                } finally {
                    mImage.close();
                    closeOutput(output);
                }
                break;
            }
            case ImageFormat.YUV_420_888:{
                ByteBuffer buffer = mImage.getPlanes()[0].getBuffer();
                byte[] bytes = new byte[buffer.remaining()];
                buffer.get(bytes);
                FileOutputStream output = null;
                try {
                    output = new FileOutputStream(mFile);
                    output.write(bytes);
                    success = true;
                } catch (IOException e) {
                    e.printStackTrace();
                } finally {
                    mImage.close();
                    closeOutput(output);
                }
                break;
            }
            default:
                Log.e(TAG, "Cannot save image, unexpected image format:" + format);

        }
        // Decrement reference count to allow ImageReader to be closed to free up resources.
        mReader.close();

        // If saving the file succeeded, update MediaStore.
        if (success) {
            MediaScannerConnection.scanFile(mContext, new String[] { mFile.getPath()},
            /*mimeTypes*/ null, new MediaScannerConnection.MediaScannerConnectionClient() {
                @Override
                public void onMediaScannerConnected() {
                    // Do nothing
                }

                @Override
                public void onScanCompleted(String path, Uri uri) {
                    Log.i(TAG, "Scanned " + path + ":");
                    Log.i(TAG, "-> uri=" + uri);
                }
            });
        }
    }

I tried to save the YUV images like a JPEG, but that way I only get one plane and the saved data don't make any sense to me...

What is the correct way to save a YUV image? Convert it to RGB (what is the sense of YUV then?)? Or with YuvImage class?

Answer 1

Typically, you don't save a YUV image as a file, and as such there are no built in functions to do so. Moreover, there are no standard image format encodings for such YUV data. YUV is typically an intermediate form of data that is convenient for the camera pipeline and conversion into other formats afterwards.

If you're really intent on this, you can write the buffers for the three channels as unencoded byte data to a file and then open it elsewhere and reconstruct it. Make sure you save the other important information too, though, such as stride data. This is what I do. Here are the relevant lines from a file format switch statement I use, along with comments on the reasoning:

File file = new File(SAVE_DIR, mFilename);
FileOutputStream output = null;
ByteBuffer buffer;
byte[] bytes;
boolean success = false;

switch (mImage.getFormat()){

    (... other image data format cases ...)

    // YUV_420_888 images are saved in a format of our own devising. First write out the
    // information necessary to reconstruct the image, all as ints: width, height, U-,V-plane
    // pixel strides, and U-,V-plane row strides. (Y-plane will have pixel-stride 1 always.)
    // Then directly place the three planes of byte data, uncompressed.
    //
    // Note the YUV_420_888 format does not guarantee the last pixel makes it in these planes,
    // so some cases are necessary at the decoding end, based on the number of bytes present.
    // An alternative would be to also encode, prior to each plane of bytes, how many bytes are
    // in the following plane. Perhaps in the future.
    case ImageFormat.YUV_420_888:
        // "prebuffer" simply contains the meta information about the following planes.
        ByteBuffer prebuffer = ByteBuffer.allocate(16);
        prebuffer.putInt(mImage.getWidth())
        .putInt(mImage.getHeight())
        .putInt(mImage.getPlanes()[1].getPixelStride())
        .putInt(mImage.getPlanes()[1].getRowStride());

        try {
            output = new FileOutputStream(file);
            output.write(prebuffer.array()); // write meta information to file
            // Now write the actual planes.
            for (int i = 0; i<3; i++){
                buffer = mImage.getPlanes()[i].getBuffer();
                bytes = new byte[buffer.remaining()]; // makes byte array large enough to hold image
                buffer.get(bytes); // copies image from buffer to byte array
                output.write(bytes);    // write the byte array to file
            }
            success = true;
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            Log.v(appFragment.APP_TAG,"Closing image to free buffer.");
            mImage.close(); // close this to free up buffer for other images
            if (null != output) {
                try {
                    output.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
        break;
    }

Because it can be up to the device exactly how the data is interlaced, it can be challenging to extract the Y,U,V channels from this encoded information later. To see a MATLAB implementation of how to read and extract a file like this, see here.

Answer 2

If you have the YuvImage object then you can convert it to Jpeg using the compressToJpeg function like so.

ByteArrayOutputStream out = new ByteArrayOutputStream();
YuvImage yuvImage = new YuvImage(data, ImageFormat.NV21, width, height, null);
yuvImage.compressToJpeg(new Rect(0, 0, width, height), 50, out);
byte[] imageBytes = out.toByteArray();
Bitmap image = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
iv.setImageBitmap(image);

You have to set the width and height of the image explicitly though.