1

I've fine tuned a distilgpt2 model using my own text using run_language_modeling.py and its working fine after training and run_generation.py script produces the expected results.

Now I want to convert this to a Tensorflow Lite model and did so by using the following

from transformers import *

CHECKPOINT_PATH = '/content/drive/My Drive/gpt2_finetuned_models/checkpoint-2500'

model = GPT2LMHeadModel.from_pretrained("distilgpt2")
model.save_pretrained(CHECKPOINT_PATH)
model = TFGPT2LMHeadModel.from_pretrained(CHECKPOINT_PATH, from_pt=True)

But I dont think I'm doing this right as after conversion, when I write

print(model.inputs)
print(model.outputs)

I get

None
None

But I still went ahead with the TFLite conversion using :

import tensorflow as tf

input_spec = tf.TensorSpec([1, 64], tf.int32)
model._set_inputs(input_spec, training=False)

converter = tf.lite.TFLiteConverter.from_keras_model(model)

# FP16 quantization:
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.float16]

tflite_model = converter.convert()

open("/content/gpt2-fp16.tflite", "wb").write(tflite_model)

But does not work and when using the generated tflite model I get the error:

tensorflow/lite/kernels/kernel_util.cc:249 d1 == d2 || d1 == 1 || d2 == 1 was not true.

Which I'm sure has something to to with my model not converting properly and getting None for input/output.

Does anyone have any idea how to fix this?

Thanks

StuckInPhDNoMore
  • 2,507
  • 4
  • 41
  • 73
  • May you please share tflite model? As well as add info about TF version. But as you mentioned problem maybe during keras model save. – Alex K. Jan 04 '21 at 13:16
  • 1
    Hi, @AlexK. I've determined that the error does not like with the ``None`` outputs. I'll create another question with a more updated details. If you dont mind, I'll tag you there :) Thanks – StuckInPhDNoMore Jan 04 '21 at 15:20
  • 1
    Sure lets see it! – Alex K. Jan 04 '21 at 15:32

0 Answers0