3

I'm using keras to build a deep autoencoder. I used its checkpointer to load the model and the weights but the result is always None which I think it means that the checkpoint dosen't work correctly and is not saving weights. Here is the code how I proceed:

checkpointer = ModelCheckpoint(filepath="weights.best.h5",
                               verbose=0,
                               save_best_only=True)
tensorboard = TensorBoard(log_dir='/tmp/autoencoder',
                          histogram_freq=0,
                          write_graph=True,
                          write_images=True)
input_enc = Input(shape=(input_size,))
hidden_1 = Dense(hidden_size1, activation='relu')(input_enc)
hidden_11 = Dense(hidden_size2, activation='relu')(hidden_1)
code = Dense(code_size, activation='relu')(hidden_11)
hidden_22 = Dense(hidden_size2, activation='relu')(code)
hidden_2 = Dense(hidden_size1, activation='relu')(hidden_22)
output_enc = Dense(input_size, activation='tanh')(hidden_2)
autoencoder_yes = Model(input_enc, output_enc)

autoencoder_yes.compile(optimizer='adam',
                         loss='mean_squared_error', 
                         metrics=['accuracy'])
history_yes = autoencoder_yes.fit(df_noyau_norm_y, df_noyau_norm_y,
                               epochs=200,
                                batch_size=batch_size,
                                shuffle = True,
                                validation_data=(df_test_norm_y, df_test_norm_y),
                                verbose=1, 
                                callbacks=[checkpointer, tensorboard]).history

autoencoder_yes.save_weights("weights.best.h5")
print(autoencoder_yes.load_weights("weights.best.h5"))

Can somebody help me find out a way to resolve the problem? Thanks

Vivek Kalyanarangan
  • 8,951
  • 1
  • 23
  • 42
Mari
  • 69
  • 1
  • 8

3 Answers3

4

No, your interpretation of load_weights returning None is not correct. Load weights is a procedure, it does not return anything, and if you assign the return value of a procedure to a variable, it will get the value of None.

So weight saving is probably working fine, its just your interpretation that is wrong.

Dr. Snoopy
  • 55,122
  • 7
  • 121
  • 140
  • Ok, so how can I access to the file containing the weights and re-used another time?? And I want to know: if the weights are correctly saved is it necessary to keep the script of the training process of the autoencoder because I'm working in the same file and I don't want each time I run the script the autoencoder starts training again ?? – Mari Mar 28 '18 at 12:59
  • @Mari You just use load_model to load the model and weights from a .hdf5 file. – Dr. Snoopy Mar 28 '18 at 16:06
3

you should use save_weights_only=True. Without this the whole model is saved not just the weights. To be able to load weights you must save weights like this:

checkpointer = ModelCheckpoint(filepath="weights.best.h5",
                           verbose=0, save_weights_only=True,
                           save_best_only=True)
Ioannis Nasios
  • 8,292
  • 4
  • 33
  • 55
0

This is expected behavior not an error. The autoencoder_yes.load_weights("weights.best.h5") doesn't actually return anything, so if you try to print the output of this function you will get None as output.

Expected behavior

In the code that you have provided, you have trained the model and saved the weights. So, the autoencoder_yes is a keras.Model object that has the fine-tuned weights.

In the same script if you load the saved weights once again, nothing is supposed to happen, the weights that you saved will get loaded again.

For clarity

Start with another fresh script, build the same model architecture and reload the weights from the h5 file and then do some predictions. In that case it will silently load the pre-trained weights and do the predictions according to that.

Vivek Kalyanarangan
  • 8,951
  • 1
  • 23
  • 42
  • Ok, so how can I access to the file containing the weights and re-used another time?? And I want to know: if the weights are correctly saved is it necessary to keep the script of the training process of the autoencoder because I'm working in the same file and I don't want each time I run the script the autoencoder starts training again ?? – Mari Mar 28 '18 at 13:00
  • No you dont need the training script at all. In the separate script (usually the test script or predict script), just don't `fit` the model, do everything that you have done and load the weights directly from the `h5` file instead of calling the `fit` function – Vivek Kalyanarangan Mar 29 '18 at 05:11
  • I understand it. But the problem now is that when I load the weights to get the encoded data layer (since I'm working on autoencoder) to do some operations on it, I get different results each time which I think it means that the weights are not fixed and saved correctly ! Do you have any explanation and solution for that ? – Mari Mar 29 '18 at 08:09