Android Camera2 API Image Color space

Question

I used this Tutorial to learn and try understand how to make a simple picture taking android app using the Camera2 API. I have added some snippets from the code to see if you all can help me understand some questions I have.

I am trying to find out how the image is saved as. Is it RGB, or BGR? Is it stored in the variable bytes?

ImageReader reader = ImageReader.newInstance(width,height,ImageFormat.JPEG, 1);


@Override
public void onImageAvailable(ImageReader reader) {
      Image image = null;
      try {
            image = reader.acquireLatestImage();
            ByteBuffer buffer = image.getPlanes()[0].getBuffer();
            byte[] bytes = new byte[buffer.capacity()];
            buffer.get(bytes);
            save(bytes);
      }

Actually, you seem to be asking about the image format, not about the colorspace. — Alex Cohn, Dec 16 '20 at 10:07

Alex Cohn · Accepted Answer · 2020-12-16T10:46:32.330

1

The image is received in JPEG format (as specified in the first line). Android uses YUV (to be more exact, YCbCr) color space for JPEG. Jpeg size is variable, it is compressed with lossy compression, and you have very little control over the level of compression.

Normally, you receive a JPEG buffer in onImageAvailable() and decode this JPEG to receive a Bitmap. You can get pixels of this Bitmap as an int array of packed SRGB pixels. The format for this array will be ARGB_8888. You don't need JNI to convert it to BGR, see this answer.

You can access Bitmap objects from C++, see ndk/reference/group/bitmap. There you can find the pixel format of this bitmap. If it was decoded from JPEG, you should expect this to be ANDROID_BITMAP_FORMAT_RGBA_8888.

edited Dec 16 '20 at 10:46

answered Dec 14 '20 at 19:49

Alex Cohn

56,089
9
113
307

In other words, by default the color format for jpeg is YUV? I see on the Android documentation that it says jpeg is just a compressed JPEG format...where other image formats, for example, says that FLEX_RGBA_8888 is a Multi-plane Android RGBA format. As Eddy mentions below, I understand correct, if I stick with using ImageFormat.jpeg then I need to decompress it to convert to BGR? My intention with all of this is to figure out what color format the image when captured is, then in JNI convert the color format to BGR by swapping bytes around in cpp... – Nelly Yuki Dec 15 '20 at 21:01
Also, could you please provide link where it says the color format of jpeg? I looked all of the documentation but have missed that info :( – Nelly Yuki Dec 15 '20 at 21:04
The colorspace of JPEG is a tricky topic, see e.g. [*Determining color space for JPEG*](https://stackoverflow.com/questions/50798014/determining-color-space-for-jpeg). But all color spaces except YCbCr are exotic, especially from camera. – Alex Cohn Dec 16 '20 at 10:04
You can request different [image formats](https://developer.android.com/reference/android/graphics/ImageFormat) when you [create an **ImageReader**](https://developer.android.com/reference/android/media/ImageReader#newInstance(int,%20int,%20int,%20int,%20long)), and some devices may allow some kind of RGB. Support for JPEG and YCbCr is mandated. – Alex Cohn Dec 16 '20 at 10:14
At any rate, if you receive a JPEG in **onImageAvailable()** and use **BitmapFactory.decodeByteArray()**, you receive a [**Bitmap**](https://developer.android.com/reference/android/graphics/Bitmap). You can [get pixels](https://developer.android.com/reference/android/graphics/Bitmap?getPixels(int[],%20int,%20int,%20int,%20int,%20int,%20int)) of this Bitmap as an int array of packed [SRGB](https://developer.android.com/reference/android/graphics/ColorSpace.Named?hl=en#SRGB) pixels. – Alex Cohn Dec 16 '20 at 10:28
Thank you Alex! I am going to try this today ^.^ – Nelly Yuki Dec 17 '20 at 14:21

score 0 · Answer 2 · answered Dec 15 '20 at 02:21

The variable bytes contains an entire compressed JPEG file. You need to decompress it to do anything much with it, such as with BitmapFactory.decodeByteArray or ImageDecoder (newer API levels).

It's not an uncompressed array of RGB values in any sense. If you want uncompressed data, the camera API supports the YUV_420_888 format, which will give you uncompressed 4:2:0 YUV data; still not RGB, though.

Android Camera2 API Image Color space

2 Answers2