5

I have a Keras model that I want to run on the Coral Edge TPU device. To do this, it needs to be a Tensorflow Lite model with full integer quantization. I was able to convert the model to a TFLite model:

model.save('keras_model.h5')

converter = tf.lite.TFLiteConverter.from_keras_model_file("keras_model.h5")
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

But when I run edgetpu_compiler converted_model.tflite, I get this error:

Edge TPU Compiler version 2.0.267685300
Invalid model: converted_model.tflite
Model not quantized

This is because I need to quantize the model, but I'm not sure how to do that. I found this page which tells me how to do this, but it wants me to make an input data generator. This is the example it provides:

def representative_dataset_gen():
  for _ in range(num_calibration_steps):
    # Get sample input data as a numpy array in a method of your choosing.
    yield [input]

How can I adapt this code to work with my input data? Where does num_calibration_steps come from? Is there a better way to do this? (I saw references to tf.contrib.tpu.keras_to_tpu_model but it has been deprecated)

Merlin04
  • 1,837
  • 4
  • 23
  • 27

1 Answers1

3

I believe num_calibration_steps is just the number of times the converter uses your rep set to determine the quantization levels. Just a guess, but maybe it subsamples from your rep set multiple times (bootstrapping or jackknifing). I'm still investigating the whole process myself, but it seems to work for me if I just pass it a single image for each yield, and use num_calibration_steps at about 100 (e.g., 100 representative images). You can see my demo script on github.

The key part is:

image_shape = (56, 56, 32)

def representative_dataset_gen():
    num_calibration_images = 10
    for i in range(num_calibration_images):
        image = tf.random.normal([1] + list(image_shape))
        yield [image]

Also see my similar response to this question: Post-training full integer quantization of Keras model

Community
  • 1
  • 1
mattroos
  • 125
  • 1
  • 11
  • 1
    I did a little more testing. It seems that representative_dataset_gen() must yield one and only one image, in the format I described. So no bootstrapping or jackknifing. The num_calibration_steps variable might be more aptly named num_calibration_images. Though there is nothing to prevent someone for using the same image more than once, so I guess num_calibration_steps still works and is a more general description. – mattroos Nov 01 '19 at 22:40
  • Thanks. Your simple example helped me solve my bug. I initially yielded a single image either as 3D array (HxWxC) in a list (i.e. `[3d_image_array]`) or as a 4D array (NxHxWxC). But seemingly TfLite wants the generator to yield a 4D array in a list (i.e. `[4d_image_array]`) Brrr, nasty bug – Bart Jan 04 '22 at 11:26