How to convert model in eager execution to static graph and save in .pb file?

Question

Imagine that I have model (tf.keras.Model):

class ContextExtractor(tf.keras.Model):
    def __init__(self):
        super().__init__()
        self.model = self.__get_model()

    def call(self, x, training=False, **kwargs):
        features = self.model(x, training=training)
        return features

    def __get_model(self):
        return self.__get_small_conv()

    def __get_small_conv(self):
        model = tf.keras.Sequential()
        model.add(layers.Conv2D(32, (3, 3), strides=(2, 2), padding='same'))
        model.add(layers.LeakyReLU(alpha=0.2))

        model.add(layers.Conv2D(32, (3, 3), strides=(2, 2), padding='same'))
        model.add(layers.LeakyReLU(alpha=0.2))

        model.add(layers.Conv2D(64, (3, 3), strides=(2, 2), padding='same'))
        model.add(layers.LeakyReLU(alpha=0.2))

        model.add(layers.Conv2D(128, (3, 3), strides=(2, 2), padding='same'))
        model.add(layers.LeakyReLU(alpha=0.2))

        model.add(layers.Conv2D(256, (3, 3), strides=(2, 2), padding='same'))
        model.add(layers.LeakyReLU(alpha=0.2))


        model.add(layers.GlobalAveragePooling2D())

        return model

I trained it and saved it using like:

   checkpoint = tf.train.Checkpoint(
                model=self.model,
                global_step=tf.train.get_or_create_global_step())
   checkpoint.save(weights_path / f'epoch_{epoch}')

It means that I have two saved files: epoch_10-2.index and epoch_10-2.data-00000-of-00001

Now I want to deploy my model. I want to get .pb file. How can I get it? I suppose I need to open my model in graph mode, load my weights and save it in pb.file. How to do it in fact?

Answer 1

You should get the session:

tf.keras.backend.get_session()

And then freeze the model, fro example as done here https://www.dlology.com/blog/how-to-convert-trained-keras-model-to-tensorflow-and-make-prediction/

def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
    """
    Freezes the state of a session into a pruned computation graph.

    Creates a new computation graph where variable nodes are replaced by
    constants taking their current value in the session. The new graph will be
    pruned so subgraphs that are not necessary to compute the requested
    outputs are removed.
    @param session The TensorFlow session to be frozen.
    @param keep_var_names A list of variable names that should not be frozen,
                          or None to freeze all the variables in the graph.
    @param output_names Names of the relevant graph outputs.
    @param clear_devices Remove the device directives from the graph for better portability.
    @return The frozen graph definition.
    """
    from tensorflow.python.framework.graph_util import convert_variables_to_constants
    graph = session.graph
    with graph.as_default():
        freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
        output_names = output_names or []
        output_names += [v.op.name for v in tf.global_variables()]
        # Graph -> GraphDef ProtoBuf
        input_graph_def = graph.as_graph_def()
        if clear_devices:
            for node in input_graph_def.node:
                node.device = ""
        frozen_graph = convert_variables_to_constants(session, input_graph_def,
                                                      output_names, freeze_var_names)
        return frozen_graph


frozen_graph = freeze_session(K.get_session(),
                              output_names=[out.op.name for out in model.outputs])

And then save the model as .pb (shown also in the link):

tf.train.write_graph(frozen_graph, "model", "tf_model.pb", as_text=False)

If this is too cumbersome try to save the keras model as a .h5 (HDF5 type file) and then follow the instruction in the link provided.

From tensorflow docs:

Write compatible code The same code written for eager execution will also build a graph during graph execution. Do this by simply running the same code in a new Python session where eager execution is not enabled.

Also from the same page:

To save and load models, tf.train.Checkpoint stores the internal state of objects, without requiring hidden variables. To record the state of a model, an optimizer, and a global step, pass them to a tf.train.Checkpoint:

checkpoint_dir = tempfile.mkdtemp()
checkpoint_prefix = os.path.join(checkpoint_dir, "ckpt")
root = tf.train.Checkpoint(optimizer=optimizer,
                           model=model,
                           optimizer_step=tf.train.get_or_create_global_step())

root.save(checkpoint_prefix)
root.restore(tf.train.latest_checkpoint(checkpoint_dir))

I recommend you the last part of this page: https://www.tensorflow.org/guide/eager

Hope this helps.

Answer 2

Thanks @BCJuan for information, I found solution.

Everyone who is looking for answer on my questions, please, look below.

NOTE: I suppose you've already saved model in checkpoint_dir and want to get this model in graph mode so that you may save it as .pb file.

model = ContextExtractor()

predictions = model(images, training=False)

checkpoint = tf.train.Checkpoint(model=model, global_step=tf.train.get_or_create_global_step())
status = checkpoint.restore(tf.train.latest_checkpoint(checkpoint_dir))
status.assert_consumed()

with tf.Session() as sess:
    status.initialize_or_restore(sess) # this is the main line for loading

    # Actually, I don't know it is necessary to pass one batch for creating graph or not   
    img_batch = get_image(...) 
    ans = sess.run(predictions, feed_dict={images: img_batch})

    frozen_graph = freeze_session(sess, output_names=[out.op.name for out in model.outputs])

# save your model
tf.train.write_graph(frozen_graph, "where/to/save", "tf_model.pb", as_text=False)