0

I need to write an Android app that, among other things, uses the Camera2 APIs to capture images in RAW format and process the resulting image data in the app. Other image formats such as YUV are not sufficient for my use case and true RAW images are required. I want to capture the image and immediately process it in-memory, without writing out an intermediate .dng file.

In order to do this, I need to use ImageFormat.RAW_SENSOR to get the image I want. The documentation for RAW_SENSOR states the following:

The layout of the color mosaic, the maximum and minimum encoding values of the raw pixel data, the color space of the image, and all other needed information to interpret a raw sensor image must be queried from the android.hardware.camera2.CameraDevice which produced the image.

However, the documentation for CameraDevice contains nothing about querying this information. A Google search turned up nothing helpful. I found this question with an answer that merely quotes what I quoted above and doesn't help figure out how to actually do it.

Thus I am lost. How do I query this information?

jcgoble3
  • 515
  • 1
  • 4
  • 18

1 Answers1

1

Android's RAW support is heavily based on what the Adobe DNG raw file format requires, so reading that spec can be helpful, to understand what the steps in RAW conversion actually are.

Quite a few fields in the CameraCharacteristics and CaptureResult objects are needed to interpret the raw buffer. The majority of the fields that start with SENSOR_ are required for processing.

See the list for the RAW capability as well, though that's still fairly vague.

The Android compliance tests include a very simple RAW processor, so you can also inspect it to see what it reads in.

Eddy Talvala
  • 17,243
  • 2
  • 42
  • 47
  • Thanks. I looked through the CameraCharacteristics and CaptureResult fields, focusing on the SENSOR_* fields, and that helped a bit. The compliance tests had a nice header comment detailing the process, and I can see the data being queried, but I was hoping to actually see the conversion code too. Unfortunately after going through the code of two methods, it seems that this code is locked behind a `ScriptC_raw_converter` class that doesn't seem to exist (it's imported at the top but doesn't exist in the given directory or anywhere else I can find). Still, it's helpful. – jcgoble3 Jun 10 '20 at 14:00
  • Never mind. I eventually figured out that that class is represented by the `raw_converter.rscript` file, though I have no clue what an .rscript file is (and Google is unhelpful). – jcgoble3 Jun 10 '20 at 14:40
  • 1
    The converter is implemented in RenderScript (https://developer.android.com/guide/topics/renderscript/compute). – Eddy Talvala Jun 10 '20 at 17:26