NV21 to Bitmap on Android, Very dark image, grayscale, or yellow tint?

Question

I have been looking at converting the NV21 byte[] that I get from onPreviewFrame(). I have searched the forums and google for various solutions. I have tried RenderScripts and some other code examples. Some of them give me an image with a yellow tint, some give me an image with red and blue flipped (after I flip it back in the code, I get yellow tint back), some give me strange color features all throughout the image (almost like a negative), some give me a grayscale image, some give me an image so dark you can't really make anything out.

Since I am the one typing the question, I realize I must be the idiot in the room so we will start with this post. This particular solution gives me a very dark image, but I am not cool enough to be able to comment yet. Has anyone tried this solution or has one that produces an image with the same quality as the original NV21 format?

I need either a valid ARGB byte[] or a valid Bitmap, I can modify my project to deal with either. Just for reference I have tried these (and a few others that are really just carbon copies of these):

One solution I tried
Another solution I tried

Answer 1

If you are trying to convert YUV from camera to Bitmap, here is something you can try:

// import android.renderscript.*
// RenderScript mRS;
// ScriptIntrinsicYuvToRGB mYuvToRGB;
// Allocation yuvPreviewAlloc;
// Allocation rgbOutputAlloc;

// Create RenderScript context, ScriptIntrinsicYuvToRGB and Allocations and keep reusing them.
if (NotInitialized) {
    mRS = RenderScript.create(this).
    mYuvToRGB = ScriptIntrinsicYuvToRGB.create(mRS, Element.YUV(mRS));    

    // Create a RS Allocation to hold NV21 data.
    Type.Builder tYuv = new Type.Builder(mRS, Element.YUV(mRS));
    tYuv.setX(width).setY(height).setYuvFormat(android.graphics.ImageFormat.NV21);
    yuvPreviewAlloc = Allocation.createTyped(mRS, tYuv.create(), Allocation.USAGE_SCRIPT | Allocation.USAGE_IO_INPUT);

    // Create a RS Allocation to hold RGBA data.
    Type.Builder tRgb = new Type.Builder(mRS, Element.RGBA_8888(mRS));
    tRgb.setX(width).tRgb(height);
    rgbOutputAlloc = Allocation.createTyped(mRS, tRgb.create(), Allocation.USAGE_SCRIPT);

    // Set input of ScriptIntrinsicYuvToRGB
    mYuvToRGB.setInput(yuvPreviewAlloc);
}

// Use rsPreviewSurface as one of the output surface from Camera API.
// You can refer to https://github.com/googlesamples/android-HdrViewfinder/blob/master/Application/src/main/java/com/example/android/hdrviewfinder/HdrViewfinderActivity.java#L504
Surface rsPreviewSurface = yuvPreviewAlloc.getSurface();
...

// Whenever a new frame is available
// Update the yuv Allocation with a new Camera buffer without any copy.
// You can refer to https://github.com/googlesamples/android-HdrViewfinder/blob/master/Application/src/main/java/com/example/android/hdrviewfinder/ViewfinderProcessor.java#L109
yuvPreviewAlloc.ioReceive();
// The actual Yuv to Rgb conversion.
mYuvToRGB.forEach(rgbOutputAlloc);

// Copy the rgb Allocation to a Bitmap.
rgbOutputAlloc.copyTo(mBitmap);

// continue processing mBitmap.
...

Answer 2

When using ScriptIntrinsics I highly recommend to update to at least JellyBean 4.3 or higher (API18). Things are much easier to use than in JB 4.2 (API 17).

ScriptIntrinsicYuvToRGB is not as complicated as it seems. Especially you don´t need Type.Builder objects. Camera preview format must be NV21 !

in the onCreate()... method create the RenderScript object and the Intrinsic:

mRS = RenderScript.create(this);
mYuvToRGB = ScriptIntrinsicYuvToRGB.create(mRS, Element.U8_4(mRS));    

With your cameraPreviewWidth and cameraPreviewHeight calculate the length of the camera data byte array:

int yuvDatalength = cameraPreviewWidth*cameraPreviewHeight*3/2 ;  // this is 12 bit per pixel

You need a bitmap for output:

mBitmap = Bitmap.createBitmap(cameraPreviewWidth, cameraPreviewHeight, Bitmap.Config.ARGB_8888);

Then you create the input and output allocations (here are the changes in API18+)

yuvPreviewAlloc = Allocation.createSized(mRS, Element.U8(mRS),  yuvDatalength);

rgbOutputAlloc =  Allocation.createFromBitmap(mRS, mBitmap);  // this simple !

and set the script-input to the input allocation

mYuvToRGB.setInput(yuvPreviewAlloc);  // this has to be done only once !

In the camera loop (whenever a new frame is avaliable), copy the NV21 byte-array (data[]) to the yuvPreviewAlloc, execute the script and copy result to bitmap:

yuvPreviewAlloc.copyFrom(data);  // or yuvPreviewAlloc.copyFromUnchecked(data);

mYuvToRGB.forEach(rgbOutputAlloc);

rgbOutputAlloc.copyTo(mBitmap);

For example: on Nexus 7 (2013, JellyBean 4.3) a full HD (1920x1080) camera preview conversion takes about 7 ms.

Answer 3

I was able to get a different method working (one that was previously linked) by using the code here. But that was giving the Red/Blue color flip. So, I just rearranged the U and V lines and all was ok. This is not as fast as a RenderScript though. It would be good to have a RenderScript that functioned properly. Here is the code:

static public void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {
    final int frameSize = width * height;

    for (int j = 0, yp = 0; j < height; j++) {
        int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
        for (int i = 0; i < width; i++, yp++) {
            int y = (0xff & ((int) yuv420sp[yp])) - 16;
            if (y < 0) y = 0;
            if ((i & 1) == 0) {
                u = (0xff & yuv420sp[uvp++]) - 128; //Just changed the order
                v = (0xff & yuv420sp[uvp++]) - 128; //It was originally v then u
            }

            int y1192 = 1192 * y;
            int r = (y1192 + 1634 * v);
            int g = (y1192 - 833 * v - 400 * u);
            int b = (y1192 + 2066 * u);

            if (r < 0) r = 0; else if (r > 262143) r = 262143;
            if (g < 0) g = 0; else if (g > 262143) g = 262143;
            if (b < 0) b = 0; else if (b > 262143) b = 262143;

            rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
        }
    }
}

Any one have a RenderScript that doesn't have color tint and or flip problems?