2

When converting and doing 8-Bit quantization of a keras model I ran into a strange error that did not happen for image data sets. 

import tensorflow.python.keras.backend as K
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import load_model
import numpy as np

x_train = np.array([[0.6171875  0.59791666],[0.6171875  0.59791666],[0.6171875  0.59791666]])
y_train = np.array([[0.6171875  0.59791666],[0.6171875  0.59791666],[0.6171875  0.59791666]])


def representative_dataset_gen():
    for i in range(1):
        # Get sample input data as a numpy array in a method of your choosing.
        sample = np.array([0.5,0.6])
        sample = np.expand_dims(sample, axis=0)
        yield [sample]



model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(2,)),
  tf.keras.layers.Dense(12, activation='relu'),
  tf.keras.layers.Dense(2, activation='softmax')
])


model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=1)

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.float32
converter.inference_output_type = tf.float32
converter.representative_dataset = representative_dataset_gen

tflite_quant_model = converter.convert()

This results in the error

ValueError: Cannot set tensor: Got value of type NOTYPE but expected type FLOAT32 for input 1, name: dense_1_input

This procedure used to work when using image data but now this happens. Tried different TF version including nightly TF2.1.

Alexander
  • 1,422
  • 11
  • 15

1 Answers1

5

Apparently the problem has to do with the data type of the input tensor which was by default Float64 and not the expected Float32. Since tflite does not know about Float64 it treads it as NOTYPE which is confusing.

Supported TF Lite Types

A cast to float32 solves the problem

sample = sample.astype(np.float32)

Alexander
  • 1,422
  • 11
  • 15
  • Thanks you so much you just saved my life. Why tensorflow doesn't say that FLOAT64 is not supported and it will be raised as NOTYPE anywhere on the internet T.T. Thanks again <3 – lamhoangtung Jul 22 '20 at 14:54