How to convert .pb to TFLite format?

Question

I downloaded a retrained_graph.pb and retrained_labels.txt file of a model I trained in Azure cognitive service. Now I want to make an Android app using that model and to do so I have to convert it to TFLite format. I used toco and I am getting the following error:

ValueError: Invalid tensors 'input' were found.

I am basically following this tutorial and have problem on step 4 and direcly copy pasted the terminal code: https://heartbeat.fritz.ai/neural-networks-on-mobile-devices-with-tensorflow-lite-a-tutorial-85b41f53230c

score 15 · Answer 1 · edited Sep 26 '19 at 05:19

I am making a wild guess here, maybe you entered input_arrays=input. Which may not be true. Use this script to find the name of the input and output arrays of the frozen inference graph

import tensorflow as tf
gf = tf.GraphDef()   
m_file = open('frozen_inference_graph.pb','rb')
gf.ParseFromString(m_file.read())

with open('somefile.txt', 'a') as the_file:
    for n in gf.node:
        the_file.write(n.name+'\n')

file = open('somefile.txt','r')
data = file.readlines()
print "output name = "
print data[len(data)-1]

print "Input name = "
file.seek ( 0 )
print file.readline()

In my case they are:

output name: SemanticPredictions
input name: ImageTensor

To note: It works for TensorFlow v1.x. So do install it using `!pip install tensorflow==1.14` — Pe Dro, Apr 14 '20 at 09:39

score 11 · Answer 2 · answered Oct 31 '18 at 10:57

You can use utility tflite_convert which is the part of tensorflow 1.10 (or higher) package.

The simple use for float inference is something like:

tflite_convert \
    --output_file=/tmp/retrained_graph.tflite \
    --graph_def_file=/tmp/retrained_graph.pb \
    --input_arrays=input \
    --output_arrays=output

Where input and output - are input and ouput tensors of your tensorflow graph

score 6 · Answer 3 · answered Jun 10 '19 at 06:14

import tensorflow as tf
gf = tf.GraphDef()
m_file = open('frozen_inference_graph.pb','rb')
for n in gf.node:
    print( n.name )

first one is input_arrays last names are output_arrays (could be more than one depends on your number of output of the model)

my output

image_tensor <--- input_array
Cast
Preprocessor/map/Shape Preprocessor/map/strided_slice/stack
Preprocessor/map/strided_slice/stack_1
.
.
.
Postprocessor/BatchMultiClassNonMaxSuppression/map/
TensorArrayStack_5/TensorArrayGatherV3
Postprocessor/Cast_3
Postprocessor/Squeeze
add/y
add
detection_boxes <---output_array
detection_scores <---output_array
detection_multiclass_scores
detection_classes <---output_array
num_detections <---output_array
raw_detection_boxes
raw_detection_scores

Pe Dro · Answer 4 · 2020-05-28T04:05:27.873

Most of the answers here prove to be broken due to the version issues. This worked for me:

Note: First find the name of the input and output layers using Netron, as I mentioned here. In my case they are input and output.

!pip install tensorflow-gpu==1.15.0

# Convert
!toco --graph_def_file /content/yolo-v2-tiny-coco.pb \
    --output_file yolo-v2-tiny-coco.tflite \
    --output_format TFLITE \
    --inference_type FLOAT \
    --inference_input_type FLOAT \
    --input_arrays input \
    --output_arrays output

Also, as per zldrobit's amazing work, you can also fetch a better quantized version of this TFLite model as:


# Now let's quantize it
!toco --graph_def_file /content/yolo-v2-tiny-coco.pb \
    --output_file quantized-yolo-v2-tiny-coco.tflite \
    --output_format TFLITE \
    --inference_type FLOAT \
    --inference_input_type FLOAT \
    --input_arrays input \
    --output_arrays output \
    --post_training_quantize

score 4 · Answer 5 · answered Jul 01 '20 at 11:27

if you're using TF2 then the following will work for you to post quantized the .pb file.

import tensorflow as tf
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(
    graph_def_file = 'path/to/frozen_inference__graph.pb', 
    input_arrays = ['Input_Tensor_Name'],
    output_arrays = ['Output_Tensor_Name'] 
)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
  f.write(tflite_model)

incase if you want full int8 quantization then

import tensorflow as tf
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(
    graph_def_file = 'path/to/frozen_inference__graph.pb', 
    input_arrays = ['Input_Tensor_Name'],
    output_arrays = ['Output_Tensor_Name'] 
)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
image_shape=(input_width,input_height,no_of_channels) #change it according to your need
def representative_dataset_gen():
    for i in range(10):
        # creating fake images
        image = tf.random.normal([1] + list(image_shape))
        yield [image]

converter.representative_dataset = tf.lite.RepresentativeDataset(representative_dataset_gen)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8] # For EdgeTPU, no float ops allowed
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8

tflite_model = converter.convert()
with tf.io.gfile.GFile('model.tflite', 'wb') as f:
  f.write(tflite_model)

score 3 · Answer 6 · answered Dec 08 '18 at 00:06

3

The error hints that you have not entered the correct

--input_arrays

From TF Lite Developer Guide I quote :

"Setting the input_array and output_array arguments is not straightforward. The easiest way to find these values is to explore the graph using TensorBoard."

Using the Tensorboard isn't hard either, by simply running this command

tensorboard --logdir=path/to/log-directory

View the TensorBoard at

localhost:6006

answered Dec 08 '18 at 00:06

Yipeng Zhang

37
3

1

where exactly in the tensorboard can we find the input_array and output_array? – Razgriz Aug 05 '19 at 10:27
@Razgriz if in your code, you did `tf.summary.FileWriter(path, sess.graph)`, you will find a tab called `GRAPHS`. Click on the variable/operation boxes in the GRAPH, you will find the name. – Zézouille Oct 10 '19 at 15:56

Sushanth · Answer 7 · 2021-01-22T16:03:31.170

To run the tflite converter on your local machine, you will need bazel and toco.

And if you read some issues in GitHub, in some versions of Tensrflow tflite causes a lot of trouble. To overcome this trouble, some recommend using tf-nightly!

To avoid all this, simply use Google Colab to convert your .pb into .lite or .tflite.

Since Colab started having the "upload" option for uploading your files into the current kernel, this I think is the most simple way without having to worry about other packages and their dependencies.

Here is the code for the same:

from google.colab import drive
drive.mount('/content/drive')


!cd drive/My\ Drive


from google.colab import files
pbfile = files.upload()


import tensorflow as tf

localpb = 'frozen_inference_graph_frcnn.pb'
tflite_file = 'frcnn_od.lite'

print("{} -> {}".format(localpb, tflite_file))

converter = tf.lite.TFLiteConverter.from_frozen_graph(
    localpb, 
    ["image_tensor"], 
    ['detection_boxes']
)

tflite_model = converter.convert()

open(tflite_file,'wb').write(tflite_model)

interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()

"""**download optimized .lite file to local machine**"""

files.download(tflite_file)

There are two ways in which you can upload your .pb file to the current session:

i) (The easy way) After running the first cell in the above notebook, the drive will be mounted. So on your left part of the screen go to the files column and right click on the folder you want to upload your .pb file and choose upload. Then use "ls" and "cd" commands to work your way into the folder and run the tflite converter cell.

ii) Run the cell with files.upload() command and click on browse and choose the .pb file from your local machine.

Once the file is uploaded, give its path to the variable "localpb" and also the name of the .lite model. Then simply run the cell having the "TFLiteConverter" comamnd.

And voila. You should have a tflite model appear in your drive. Simply right-click on it and download to your local machine to run inferences.

score 1 · Answer 8 · answered May 02 '19 at 09:58

without bazel you can try the following code

pip uninstall tensorflow
pip install tf-nightly
pip show protobuf

If protobuf is version 3.6.1, then proceed to installing the pre-release version of 3.7.0.

pip uninstall protobuf
pip install protobuf==3.7.0rc2

I still couldn’t get the command line version to work. It kept returning the error: “tflite_convert: error: –input_arrays and –output_arrays are required with –graph_def_file” although both parameters were supplied. It worked in Python, however.

import tensorflow as tf

graph_def_file = "model.pb"
input_arrays = ["model_inputs"]
output_arrays = ["model_outputs"]

converter = tf.lite.TFLiteConverter.from_frozen_graph(
        graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

Hi Zeshan, I'm also doing same but getiing error like below. Please help Traceback (most recent call last): File "freeze_graph.py", line 8, in converter = tf.lite.TFLiteConverter.from_frozen_graph( AttributeError: type object 'TFLiteConverterV2' has no attribute 'from_frozen_graph' — Prashant Gaikwad, Sep 13 '19 at 11:07

score 1 · Answer 9 · answered Aug 30 '19 at 21:54

Substituting Mul for input fixed it for me.

IMAGE_SIZE=299
tflite_convert \
  --graph_def_file=tf_files/retrained_graph.pb \
  --output_file=tf_files/optimized_graph.lite \
  --input_format=TENSORFLOW_GRAPHDEF \
  --output_format=TFLITE \
  --input_shape=1,${IMAGE_SIZE},${IMAGE_SIZE},3 \
  --input_array=Mul \
  --output_array=final_result \
  --inference_type=FLOAT \
  --input_data_type=FLOAT

score 1 · Answer 10 · answered Sep 20 '19 at 08:03

I am following up from my previous answer you can use the following script to convert your trained model on ssd mobilenet to tflte using

python object_detection/export_tflite_ssd_graph \
    --pipeline_config_path ssd_0.75_export/pipeline.config \
    --trained_checkpoint_prefix ssd_0.75_export/model.ckpt \
    --output_directory ssd_to_tflite_output

To do this you will first need to be present in research folder of tensorflow object detection API, and change the dile path/name as per your names. If this dosent work try running this script from research folder and rerun:

protoc object_detection/protos/*.proto --python_out=.
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

score 0 · Answer 11 · answered Nov 28 '18 at 19:58

Most likely it's because during the retraining process the input and output tensors were renamed. If this is a retrained inceptionv3 graph, try using Mul as the input tensor name and final_result as the output tensor name.

bazel run --config=opt //tensorflow/contrib/lite/toco:toco -- \
    ... other options ...
    --input_shape=1,299,299,3 \
    --input_array=Mul \
    --output_array=final_result

Similar adjustment if you use tflife_convert as Aleksandr suggest.

score 0 · Answer 12 · edited Jul 24 '19 at 09:08

0

import tensorflow as tf

!tflite_convert \
--output_file "random.tflite" \
--graph_def_file "pb file path" \
--input_arrays "input tensor name" \
--output_arrays "output tensor name"

edited Jul 24 '19 at 09:08

HMD

2,202
6
24
37

answered Jul 24 '19 at 09:03

Intern

1
1

How to convert .pb to TFLite format?

12 Answers12

Linked