Tflite int8 Quantise model with multiple inputs and variable output

Question

Hi I have a neural network that takes in 2 inputs using the Functional API

varyNetworkInput = tf.keras.Input(shape=(1,), batch_size=1)

mainInput = tf.keras.Input(shape=(10, 10))

...

x = tf.keras.layers.Lambda(switcher)([varyNetworkInput, mainInput])

...

# (Speculation) Since the output from path1 and path2 are different
# I am flattening the output so that the rank of the output will be the same
# I am speculating this could help reduce errors with quantisation, though not certain
output = tf.keras.layers.Flatten()(x)   

model = tf.keras.Model(inputs=[varyNetworkInput, mainInput], outputs=output)

varyNetworkInput: Is either 0 or 1. This indicates what path the data should pass down the neural network taken by the network

mainInput: Is the main data being inputted

I have a lambda function that changes the path of the neural network

def switcher(inputs):
    varyNetworkInput = inputs[0]
    mainInput = inputs[1]
    pathValue = varyNetworkInput[0]
    return tf.cond(
        pathValue == 1,
        lambda: path1(mainInput),
        lambda: path2(mainInput)
    )

I want to int8 quantise the neural network using:

converter = tf.lite.TFLiteConverter.from_keras_model(model)

converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_quant_model_bytes = converter.convert()
open("int8QuantisedModel.tflite", "wb").write(tflite_quant_model_bytes)

I am currently getting this error:

RuntimeError: Attempting to resize dimension 3 of tensor 1 with value 1 to 3. ResizeInputTensorStrict only allows mutating unknown dimensions identified by -1.

I have successfully been able to quantise without int8 with just these settings

converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset

However, for my specific use case I require int8

Edit

Ok I made a very dumb mistake, the data I was using in representative_dataset had the wrong dimensions

Tflite int8 Quantise model with multiple inputs and variable output

Edit

0 Answers0