Grayscale a camera image in Flutter for tflite model input

Question

I have created a tflite model for sign language. The input images for the model are required to be grayscaled. In the app, I take images from the camera stream and want to feed them to the model for inference. The images come in the format bgra in iPhone, however all my tries to convert the color channel from three to one has failed.

The main problem is that I can only get an bgra formatted image from the fromBytes() method no matter what I try. And then I cannot reformat that image in any way.

Get image from camera Image and grayscale after. Grayscale function does not work, because it does not accept bgra formatted images.

/// Converts a [CameraImage] in BGRA888 format to [imageLib.Image] in grayscale format
static imageLib.Image convertBGRA8888ToImage(CameraImage cameraImage) {
    int width = cameraImage.planes[0].width!;
    int height = cameraImage.planes[0].height!;
    Uint8List bgraData = cameraImage.planes[0].bytes;
    Uint8List grayData = bgraToGrayscale(bgraData, width, height);
    imageLib.Image img = imageLib.Image.fromBytes(width, height, grayData,
        format: imageLib.Format.bgra);

    img = imageLib.grayscale(img);

    return img;
}

Convert bytes before to one channel. The I use the same method from above and change format to luminance, but that format is not accepted.

// Define a function to convert a BGRA8888 image to grayscale
static Uint8List bgraToGrayscale(Uint8List bgraData, int width, int height) {
    // Allocate a new array to store the grayscale data
    Uint8List grayData = Uint8List(width * height);

    // Iterate through each pixel in the image
    for (int y = 0; y < height; y++) {
      for (int x = 0; x < width; x++) {
        // Calculate the offset of the current pixel in the BGRA data
        int offset = (y * width + x) * 4;

        // Extract the blue, green, and red components of the pixel
        int blue = bgraData[offset];
        int green = bgraData[offset + 1];
        int red = bgraData[offset + 2];

        // Calculate the grayscale value of the pixel (using the luminance method)
        int gray = (0.2126 * red + 0.7152 * green + 0.0722 * blue).round();

        // Store the grayscale value in the output array
        grayData[y * width + x] = gray;
      }
    }

    // Return the grayscale image data
    // grayData.length == bgraData.length / 4
    return grayData;
}

In the end I make the image a TensorImage, but I also would not have any idea how to change the TensorImage to grayscale if that is even possible.

final inputTensor = TensorImage(_model.inputType);
inputTensor.loadImage(image);

Thank you for any possible solutions!

Grayscale a camera image in Flutter for tflite model input

0 Answers0