0

I am new to tensorflow and keras. I trained a CNN for sentence classification using keras and exported the model using following code

K.set_learning_phase(0)
config = model.get_config()
weights = model.get_weights()

new_model = Sequential.from_config(config)
new_model.set_weights(weights)

builder = saved_model_builder.SavedModelBuilder(export_path)
signature = predict_signature_def(
    inputs={'input': new_model.inputs[0]},
    outputs={'prob': new_model.outputs[0]})


with K.get_session() as sess:

    builder.add_meta_graph_and_variables(
        sess=sess,
        tags=[tag_constants.SERVING],
        clear_devices = True,
        signature_def_map={
            signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature}
    )
builder.save()

I got variables.data-00000-of-00001 and variables.index in variables folder and saved_model.pb.

I want to combine these files into one file before deploying for prediction. In the end I want to quantize the model as variables file size is really huge and I think before using the quantize functionality from tensorflow I need to have my model frozen in a pb file. Please help

manoveg
  • 423
  • 1
  • 3
  • 13

2 Answers2

0

You can use the freeze_graph.py tool to combine your files into a single file.

This will output a single GraphDef file that holds all of the weights and architecture.

You'd use it like this:

bazel build tensorflow/python/tools:freeze_graph && \
bazel-bin/tensorflow/python/tools/freeze_graph \
--input_graph=some_graph_def.pb \
--input_checkpoint=model.ckpt-8361242 \
--output_graph=/tmp/frozen_graph.pb --output_node_names=softmax

Where input_graph is your saved_model.pb file.

And where input_checkpoint are your variables in your variables folder, and they might look like this:

/tmp/model/model-chkpt-8361242.data-00000-of-00002
/tmp/model/model-chkpt-8361242.data-00001-of-00002
/tmp/model/model-chkpt-8361242.index
/tmp/model/model-chkpt-8361242.meta

Note that you refer to the model checkpoint as model-chkpt-8361242 in this case, for instance.

You take the prefix of each of the files you have there when using the freeze_graph.py tool.

Clarence Leung
  • 2,446
  • 21
  • 24
  • 2
    I tried but getting following error `raise self.ParseError('Expected identifier or number, got %s.' % result) google.protobuf.text_format.ParseError: 1:1 : Expected identifier or number, got.` – manoveg Nov 28 '17 at 01:07
0

how are you planning to serve your model? TensorFlow Serving supports the SavedModelFormat natively - without requiring the freeze_graph.py step.

if you still want to manually combine the graph and the variables (and use freeze_graph.py), you'll likely need to use the older ExportModel format as Clarence demonstrates above.

also, you'll likely want to switch to the Estimator API at this point, as well.

here are some examples using all of the above: https://github.com/pipelineai/pipeline

Chris Fregly
  • 1,490
  • 1
  • 12
  • 8
  • 1
    basically I want to deploy it on google cloud ml engine and there the maximum model size limit is 250 MB. My model size is bigger than this. Therefore I want to quantize it. What I understand so far is that I want the frozen .pb file before I can try quantizing. Correct me if I am wrong. – manoveg Nov 29 '17 at 10:56