0

So, I was wondering how to load the latest checkpoint in Tensorflow having its path/directory and continue the training where I left off. And also how to load the latest checkpoint and save it as a complete model. Please help me

My code:

        cp_callback = tf.keras.callbacks.ModelCheckpoint(
            filepath=checkpoint_path, 
            verbose=1, 
            save_weights_only=True,
            save_freq=1*batch_size)

        # Create a basic model instance
        model = create_model(training, output)
        model.save_weights(checkpoint_path.format(epoch=0))

        # Create a TensorBoard callback (for metrics monitoring)
        tb_callback = tf.keras.callbacks.TensorBoard(log_dir="chatbot/training/logs", histogram_freq=1, update_freq= 1, profile_batch= 1)

        # Train the model with the new callback
        model.fit(training, output, epochs=500, batch_size = batch_size, validation_data=(training, output), callbacks=[cp_callback, tb_callback], verbose = 1)
Dweller
  • 9
  • 5

1 Answers1

2

The simplest solution to your problem would be to save the entire model with the ModelCheckpoint callback. You only have to remove the save_weights_only argument for it to work.

cp_callback = tf.keras.callbacks.ModelCheckpoint(
            filepath=checkpoint_path, 
            verbose=1,
            save_freq=1*batch_size)

To load the checkpoint and continue training at a later point in time, just call

model = tf.keras.models.load_model(checkpoint_path)

If you want to load a checkpoint given you only saved the model weights, you have to first build your model and transfer your saved weights into it.

model.load_weights(checkpoint_path)

If you need further information about loading and saving models, I would recommend reading the documentation: https://www.tensorflow.org/guide/keras/save_and_serialize

This answer is referencing the answer of : Save and load weights in keras

nFinite
  • 31
  • 2
  • so, it is like `model = create_model()` then `model.load_weights(checkpoint_path)` and the `model.fit()`? – Dweller Nov 05 '22 at 15:29
  • As long as the model created by `create_model()` has the same architecture as the model from which you saved the weights, it should work – nFinite Nov 05 '22 at 15:33
  • im saving a model using `model.save()`, so should just use `create_model()` and then load weights and save it? or how should i save the entire model, i dont undersatnd @nFininte – Dweller Nov 05 '22 at 15:49
  • If you save your model with `model.save()` it will save your architecture, weights, compilation information and optimizer state. The best/easiest thing for you would be to save the entire model instead of only saving the weights with the ModellCheckpoint callback and reload your saved model with `model = tf.keras.models.load_model()` whenever you want to continue training or use it for inference. – nFinite Nov 05 '22 at 16:00