Is it possible to convert a TensorFlow (Keras) model from BGR to RGB?

Question

I have converted a Caffe model, learned on BGR data, to ONNX format and then from ONNX to TensorFlow (Keras) So now I have a Keras model, learned on BGR data. Is it possible to convert it in such a way, that it will properly work with RGB data?

I've tried to convert it to OpenVINO with the --reverse_input_channel flag and then back to TensorFlow, but it seems that openvino2tensorflow works very poorly, so it didn't work. Maybe there is some simpler way?

Update

I've realized that I get SavedModel through Keras model, so I've updated the question.

Update 2

I've applied AndrzejO solution. However, now my model gives much worse results than before. Am I doing something wrong?

from keras.layers import Input, Lambda
from keras.models import Model

input_shape = k_model.get_layer(index = 0).input_shape[0][1:]
inputs = Input(shape=input_shape)
lambda_layer = Lambda(lambda x: x[:,:,:,::-1])(inputs)
outputs = k_model(lambda_layer)
k_model = Model(inputs=inputs, outputs=outputs)

Update 3

Regarding AndrdzejO hint, I've tested the reversed model on the reversed (BGR) images and compared results with the former model on normal (RGB) images. That's strange - they are similar but not the same. Below is the code (in Java) that reverses the image:

  public static byte[] getPixelsBGR(Bitmap image) {
    // calculate how many bytes our image consists of
    int bytes = image.getByteCount();

    ByteBuffer buffer = ByteBuffer.allocate(bytes); // Create a new buffer
    image.copyPixelsToBuffer(buffer); // Move the byte data to the buffer

    byte[] pixels = buffer.array(); // Get the underlying array containing the data.

    // Copy pixels into place
    for (int i = 0; i < pixels.length/4; i++)
    {
      byte pom = pixels[i*4];
      pixels[i * 4] = pixels[i * 4 + 2];
      pixels[i * 4 + 2] = pom;
    }

    return pixels;
  }

if (!modelBGR)
{
    byte[] pixels = getPixelsBGR(resizedBitmap);
    ByteBuffer pixelBuffer = ByteBuffer.wrap(pixels);
    resizedBitmap.copyPixelsFromBuffer(pixelBuffer);
}

**Update 4**

**AndrzejO**'s solution works perfectly. It correctly reverses the channel order. The thing was I had been subtracting channels mean in the tflite metadata and forgot that I need to also reverse the order of channels there. After I've corrected this, I have the exact same results, which implies that the reversion of channels has worked perfectly.

For some reason, reversing the channel order in my case makes inference less accurate (as if though the channels have alreasdy been reversed in some earlier conversion process), but that's a different thing to investigate

Not exactly a solution to the problem - more of a workaround -, but why not convert the input data instead of trying to change the model ? — SpaceBurger, Sep 21 '22 at 16:26
What is the BGR format? Is the same as RGB, just in different channel order? — AndrzejO, Sep 21 '22 at 16:41
change the channel order in the input, or, depending on the type of the first layer, you can try to swap filters order / matrix weights — Alberto Sinigaglia, Sep 21 '22 at 17:58
@SpaceBurger Because the input is not that simple to convert - it's an Android (Kotlin) project which takes input from the camera (YCrCb): https://github.com/googlesamples/mlkit/tree/master/android/material-showcase . Plus converted model would be more universal - I could use it with many different projects. — Hououin Kyouma, Sep 22 '22 at 11:07
@AlbertoSinigaglia I don't know how to properly visualize the SavedModel in the netron.app , so I visualize the tflite version. First there is an input. Then there is Pad, and then Conv2D with filter (64x7x7x3) and bias (64). I guess I would need to swap the values in that filter? Thank you - now I understand a bit better how is that working. But how can I perform that swap? Is there some tool for that? Or maybe I need to read the model, for example in Python, and somehow modify it, then export it again? — Hououin Kyouma, Sep 22 '22 at 11:28

score 1 · Accepted Answer · answered Sep 22 '22 at 12:20

1

You can create a new model: first a lambda layer which will reverse the channel order, than your saved model:

input_shape = old_model.get_layer(index = 0).input_shape[0][1:]
inputs = Input(shape=input_shape)
lambda_layer = Lambda(lambda x: x[:,:,:,::-1])(inputs)
outputs = old_model(lambda_layer)
new_model = Model(inputs=inputs, outputs=outputs)

answered Sep 22 '22 at 12:20

AndrzejO

1,502
1
9
12

Thank you! That seems to be it. However, after applying that solution, my model has started to give much worse results. Am I doing something wrong? (I've updated the question). – Hououin Kyouma Sep 22 '22 at 16:29
@HououinKyouma makes no sense, probably you are not applying some "pre-transformation" that you applied during training – Alberto Sinigaglia Sep 22 '22 at 16:45
@HououinKyouma Did you try the new model on the same data? that is, you converted your BGR images into RGB images? It should give exactly the same predictions – AndrzejO Sep 22 '22 at 16:51
@AlbertoSinigaglia here is the specification for the model inputs: "Inputs to the network should be scaled to (0,255), presented in BGR channel order and have a mean of B:104, G:117, R:124 subtracted." I've applied the means subtraction before through the tflite metadata and have had some quite decent results. The only thing I've changed now is in the code (Update 2). – Hououin Kyouma Sep 22 '22 at 19:20
@AndrzejO thank you for a good hint about what to check. I'll try to apply it. – Hououin Kyouma Sep 22 '22 at 19:20
@AndrzejO That's strange. For the exact same images, results are similar but not the same. Maybe it has something to do with data types? (Update 3) – Hououin Kyouma Sep 22 '22 at 20:45
1

Thank you very much! That is reversing the channel order perfectly! I've corrected the test and now it gives the exact same predictions. For some strange reason, reversing the channel order in my case makes inference less accurate (as if though the channels have alreasdy been reversed in some earlier conversion process), but that's a different thing – Hououin Kyouma Sep 23 '22 at 14:19
1

@AlbertoSinigaglia thank you - your suggestion was partially correct - I forgot to correct the pre-transformation (order of mean channels in the tflite metadata) after reversing the images. – Hououin Kyouma Sep 23 '22 at 14:50

Is it possible to convert a TensorFlow (Keras) model from BGR to RGB?

1 Answers1