ScriptIntrinsicYuvToRGB or YUV420 Allocation is broken?

Question

I try to use ScriptIntrinsicYuvToRGB class from renderscript to make YUV to RGB conversion, where source is in YUV420 format. I have 3 raw planes which I read from files and try to feed them into YUV-kind of Allocation, and pass it through ScriptIntrinsicYuvToRGB.forEach.

It converts luma (Y plane) correctly, but fails on colors because chroma channels seem to read all values from buf[w*h] location - see commented part in code sample. It looks like bug when Allocation doesn't properly address UV planes. I assume so because I tested in a script using rsGetElementAtYuv_uchar_U function on the allocation, and it gives the same value (from buf[w*h]) for any coordinates.

I searched all places if I could further specify YUV format such as strides/offsets etc, but didn't find anything more that setting Element.DataKind.PIXEL_YUV and Type.Builder.setYuvFormat(ImageFormat.YUV_420_888).

Can someone help with this?

{
      int w = 320, h = 172;
      ScriptIntrinsicYuvToRGB yc = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));
      {
         Element elemYUV = Element.createPixel(rs, Element.DataType.UNSIGNED_8, Element.DataKind.PIXEL_YUV);
         Type typeYUV = new Type.Builder(rs, elemYUV).setX(w).setY(h).setYuvFormat(ImageFormat.YUV_420_888).create();
         Allocation yuv = Allocation.createTyped(rs, typeYUV);
         byte[] buf = new byte[yuv.getBytesSize()];
         int offs = 0;
         for(int i=0; i<3; i++){
            int sz = w*h;
            if(i>0)
               sz /= 4;
            InputStream is = new FileInputStream("/sdcard/yuv/"+(i==0 ? 'y' : i==1 ? 'u' : 'v'));
            int n = is.read(buf, offs, sz);
            if(n!=sz)
               throw new AssertionError("!");
            offs += sz;
            is.close();
         }
//               buf[w*h] = 0x40;
         yuv.copyFrom(buf);
         yc.setInput(yuv);
      }

      Type outType = new Type.Builder(rs, Element.U8_4(rs)).setX(w).setY(h).create();
      Allocation out = Allocation.createTyped(rs, outType);
      yc.forEach(out);

      int[] buf = new int[out.getBytesSize()/4];
      out.copy1DRangeToUnchecked(0, w*h, buf);

      bm = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888);
      bm.setPixels(buf, 0, bm.getWidth(), 0, 0, bm.getWidth(), bm.getHeight());
      iv.setImageBitmap(bm);

      yc.destroy();
   }

score 0 · Answer 1 · answered Sep 18 '20 at 15:34

I believe that you need setYuvFormat() for your Type. Here are the two lines that I use to build my Allocation:

Type typeYUV = Type.Builder(rs, Element.YUV(rs)).setYuvFormat(ImageFormat.NV21).create();
Allocation yuv = Allocation.createSized(rs, typeYUV.element, width*height*3/2);

sakridge · Answer 2 · 2016-12-17T20:28:26.857

One solution is to just fill in a U8 allocation and do the indexing yourself in a custom script:

#pragma rs_fp_relaxed
rs_allocation yuv_in;
uint32_t width;
uint32_t offset_to_u;
uint32_t offset_to_v;
uchar4 RS_KERNEL yuv_to_rgba(uint32_t x, uint32_t y) {
    uint32_t index = y * width + x;
    uint32_t uv_index = (y >> 1) * width + (x >> 1);
    float Y = (float)rsGetElementAt_uchar(yuv_in, index);
    float U = (float)rsGetElementAt_uchar(yuv_in, uv_index + offset_to_u);
    float V = (float)rsGetElementAt_uchar(yuv_in, uv_index + offset_to_v);
    float3 f_out;
    f_out.r = Y + 1.403f * V;
    f_out.g = Y - 0.344f * U - 0.714f * V;
    f_out.b = Y + 1.770f * U;
    f_out = clamp(f_out, 0.f, 255.f);
    uchar4 out;
    out.rgb = convert_uchar3(f_out);
    out.a = 255;
    return out;
}

java:

sc.set_yuv_in(yuv_allocation);
sc.set_width(width);
sc.set_offset_to_u(width * height);
sc.set_offset_to_v(width * height + (width/2 * height/2));
sc.forEach_yuv_to_rba(out);

YUV_420_888 is more of a generic YUV type for importing from other YUV resources in android. I don't think there is a way to set the stride/offset values for the u/v planes to make it useful for a custom conversion.

ScriptIntrinsicYuvToRGB or YUV420 Allocation is broken?

2 Answers2