1

I'm setting up a custom training environment in tensorflow-keras, and I want to know if is possible to reconnect shared weights of models that were saved onto different files.

I have an attention encoder-decoder model, and as is known, the training model of attention is a little bit different from the predict model, but those models are sharing the same weights. First i save a non-trained model into 3 files with tf.keras.models.save_model:

  • Full Model ( Training Model )
  • Encoder Model ( Prediction Model )
  • Decoder Model ( Prediction Model )

and then, I try to use tf.keras.models.load_model to load the three above models and train only the full model as usual.

full_model = tf.keras.models.load_model(
    'full_model.h5'), 
    custom_objects={'AttentionLayer': AttentionLayer}
)
encoder_model = tf.keras.models.load_model(
    os.path.join('encoder.h5'), 
    custom_objects={'AttentionLayer': AttentionLayer}
)
decoder_model = tf.keras.models.load_model(
    os.path.join('decoder.h5'), 
    custom_objects={'AttentionLayer': AttentionLayer}
)

full_model.train()...

So, the full_model weights are updating as expected... but, the encoder and decoder weights are still freezing. Is there some way to reconnect the graph of those models?

Jose Daniel
  • 11
  • 3
  • You can copy the layer weights from the trained `full_model` to the not-yet-trained encoder and decoder. Have a look at [this question](https://stackoverflow.com/questions/48547688/tensorflow-keras-copy-weights-from-one-model-to-another). – wohe1 Sep 26 '19 at 05:42
  • Was thinking on that, but the full model have 22 weights, the encoder have 13 and the decoder have 9. They are not ordered with respect to each other. – Jose Daniel Sep 26 '19 at 09:30
  • So are you saying that the full model is completely different from the encoder and decoder? In that case it wouldn't make sense to share weights between those models... – wohe1 Sep 26 '19 at 09:35
  • Not the best expected answer, but I think you should rebuild the full model from the other models instead of loading it. – Daniel Möller Sep 26 '19 at 13:22
  • Wohe, my full model have shared weights with encoder model and decoder model, but when i open it the weights variables are unordered, so i cant create a function that allows me to build without do it manually. – Jose Daniel Sep 26 '19 at 17:03
  • Daniel, i think in the next version i will take your advice. Thanks. – Jose Daniel Sep 26 '19 at 17:03

0 Answers0