OutOfMemory Error in processing frames from onPreviewFrame

Question

I am capturing frames in OnPreviewFrame() and then processing them in a thread to check if they are valid or not.

public void onPreviewFrame(byte[] data, Camera camera) {
    if (imageFormat == ImageFormat.NV21) {
        //We only accept the NV21(YUV420) format.
        frameCount++;
        if (frameCount > 19 && frameCount % 2 == 0) {
            Camera.Parameters parameters = camera.getParameters();
            FrameModel fModel = new FrameModel(data);
            fModel.setPreviewWidth(parameters.getPreviewSize().width);
           fModel.setPreviewHeight(parameters.getPreviewSize().height);
            fModel.setPicFormat(parameters.getPreviewFormat());
            fModel.setFrameCount(frameCount);
            validateFrame(fModel);
           }
      }       
  }

In validateFrame(), i submit a ValidatorThread runnable instance to a ThreadPoolExecutor with 4 core and max threads, to process the frames parallelly.

public class ValidatorThread implements Runnable {

private FrameModel frame;

public ValidatorThread(FrameModel fModel) {
    frame = fModel;
}

@Override
public void run() {
    try {
        Thread.sleep(1000);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    processNV21Data();
}

private void processNV21Data() {

    YuvImage yuv = new YuvImage(frame.getData(), frame.getPicFormat(),
            frame.getPreviewWidth(), frame.getPreviewHeight(), null);
    frame.releaseData();

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    yuv.compressToJpeg(new Rect(0, 0, frame.getPreviewWidth(), frame.getPreviewHeight()), 100, out);

    byte[] bytes = out.toByteArray();
    yuv = null;

    try {
        if (out != null)
            out.close();
        out = null;
    } catch (IOException e) {
        e.printStackTrace();
    }

    Bitmap baseBitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
    bytes = null;

    // rotate bitmap 
    baseBitmap = rotateImage(baseBitmap, frame.getRotation());

    //create copy of original bitmap to use later
    Bitmap mCheckedBitmap = baseBitmap.copy(Bitmap.Config.ARGB_8888, true);

    // convert base bitmap to greyscale for validation
    baseBitmap = toGrayscale(baseBitmap);

    boolean isBitmapValid =  Util.isBitmapValid(baseBitmap);

    if (isBitmapValid) {
        baseBitmap.recycle();
        mCheckedBitmap.recycle();
        frame = null;
    } else {
        baseBitmap.recycle();
        mCheckedBitmap.recycle();
        frame = null;
    }
}

public Bitmap toGrayscale(Bitmap bmpOriginal) {
    int width, height;
    height = bmpOriginal.getHeight();
    width = bmpOriginal.getWidth();
    Bitmap bmpGrayscale = Bitmap.createBitmap(width, height, Bitmap.Config.RGB_565);
    Canvas c = new Canvas(bmpGrayscale);
    Paint paint = new Paint();
    bmpOriginal.recycle();
    return bmpGrayscale;
}
private Bitmap rotateImage(final Bitmap source, float angle) {
    Matrix matrix = new Matrix();
    matrix.postRotate(angle);
    Bitmap rotatedBitmap = Bitmap.createBitmap(source, 0, 0, source.getWidth(), source.getHeight(), matrix, true);
    source.recycle();
    return rotatedBitmap;
}

}

The FrameModel class has such declaration :

public class FrameModel {

private byte[] data;
private int previewWidth;
private int previewHeight;
private int picFormat;
private int frameCount;

 public void releaseData() {
    data = null;
}

// getters and setters
}

I am getting OutOf Memory error while processing multiple frames.

Can anyone help what memory optimisation does the code need?

Probably the frames captured have a big resolution. Try to resize the YuvImage and the Bitmap before processing it — MatPag, Jun 19 '17 at 06:45
The heap is not getting deallocated.. Is there any memory leak in the code here ? — Manvendra Sah, Jun 19 '17 at 07:19
Have you tried to set to null the Bitmaps at the end of the method? — MatPag, Jun 19 '17 at 07:31
Sometimes i noticed that recycle didn't work really well when really fast GC is occurring — MatPag, Jun 19 '17 at 09:57
I can use inBitmap option with the decodeByteArray() function to save OOM. But when i am rotating and grey-scaling the image, i am using createBitmap().. This is causing problems.. How do i avoid OOM, because createBitmap() gives no option to reuse bitmaps... — Manvendra Sah, Jun 20 '17 at 15:54
Used inBitmap and created a bitmapPool to solve my problems. Thanks. — Manvendra Sah, Jun 28 '17 at 05:27

Alex Cohn · Answer 1 · 2017-06-21T08:29:34.473

0

You can reduce memory usage if you produce grayscale bitmap from YUV data without going through Jpeg. This will also be significantly faster.

public Bitmap yuv2grayscale(byte[] yuv, int width, int height) {
    int[] pixels = new int[width * height];

    for (int i = 0; i < height*width; i++) {
        int y = yuv[i] & 0xff;
        pixels[i] = 0xFF000000 | y << 16 | y << 16 | y;
    }
    return Bitmap.createBitmap(pixels, width, height, Bitmap.Config.RGB_565);
}

Alternatively, you can create an RGB_565 bitmap without going through int[width*height] pixels array, and manipulate the bitmap pixels in place using NDK.

edited Jun 21 '17 at 08:29

answered Jun 19 '17 at 07:58

Alex Cohn

56,089
9
113
307

this is returning int[].. can i get the greyscaled data in byte[].. because i want to use Bitmap.decodeByteArray() rather then createbitmap().. – Manvendra Sah Jun 21 '17 at 07:27
decodeByteArray() works on compressed stream (see *[Formats supported by BitmapFactory.decodeByteArray(…)](https://stackoverflow.com/questions/24333450/formats-supported-by-bitmapfactory-decodebytearray)*), but the camera produces YUV. To compress it to JPEG and decompress back to Bitmap, you waste both time and memory. – Alex Cohn Jun 21 '17 at 08:18

OutOfMemory Error in processing frames from onPreviewFrame

1 Answers1