tflite uint8 quantization model input and output float conversion

Question

I have successfully converted a quantized 8bit tflite model for object detection. My model was originally trained on images that are normalized by dividing 255 so the original input range is [0, 1]. Since my quantized tflite model requires input to be uint8, how can I convert my image (originally [0, 255]) to be correct for my network? Also how can I convert output to float to compare the results with floating point model?

The following code does not give me the right result.

'''python
im = cv2.imread(image_path)
im = im.astype(np.float32, copy=False)
input_image = im
input_image = np.array(input_image, dtype=np.uint8)
input_image = np.expand_dims(input_image, axis=0)

interpreter.set_tensor(input_details[0]['index'], input_image)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
output_data2 = interpreter.get_tensor(output_details[1]['index'])
output_data3 = interpreter.get_tensor(output_details[2]['index'])

min_1 =  -8.198164939880371
max_1 = 8.798029899597168
scale = (max_1 - min_1)/ 255.0

min_2 = -9.77856159210205
max_2 = 10.169703483581543
scale_2 = (max_2 - min_2) / 255.0

min_3 = -14.382895469665527 
max_3 = 11.445544242858887
scale_3 = (max_3 - min_3) / 255.0

output_data = (output_data ) * scale + min_1
output_data2  = (output_data2) * scale_2 + min_2
output_data3 = (output_data3) * scale_3 + min_3

'''

For input, it should be `input_image = im / 256` But for output, where do you take these min/max values from? — Alex Cohn, Jul 24 '19 at 18:31
it's from the fakequantization node added by tf.lite.create_train_graph() that keeps track of each layer's min max activation for quantization — Jasmine Liu, Jul 24 '19 at 20:46
The values that you happen to get for output in certain case may not correct for scaling the output to **unit8**. — Alex Cohn, Jul 24 '19 at 20:48
I'm not sure if I understand you correctly. But tflite quantization keeps track of min and max value and perform a uniform quantization over the range, so the floating output should be correctly represented in uint8. I test the floating point model with the fake quantization nodes and the output is correct. — Jasmine Liu, Jul 24 '19 at 22:07
maybe if you provide correctly scaled inputs, the outputs will become better — Alex Cohn, Jul 24 '19 at 22:27

score 0 · Answer 1 · answered Sep 23 '19 at 10:02

i met the same problem but in pose estimation.

have you solved the problem yet?

you use quantized aware training?

i think you can get a q and z value(because you have to give mean and std-err when you use tflite api or toco commonad to get a quantized 8bit tflite model) about your input image.

try these codes:

  image = q_input* (image - z_input)

  output_data  = q_output(image - z_output)

etc.

(for different layers you can access different q and z)

Let me know if you tried this way

score 0 · Answer 2 · answered Jul 01 '20 at 22:58

I've converted the image via OpenCV to "CV_8UC3" and this worked for me:

  // Convert to RGB color space
  if (image.channels() == 1) {
    cv::cvtColor(image, image, cv::COLOR_GRAY2RGB);
  } else {
    cv::cvtColor(image, image, cv::COLOR_BGR2RGB);
  }

  image.convertTo(image, CV_8UC3);

tflite uint8 quantization model input and output float conversion

2 Answers2