6

I want to use Keras ModelCheckpoint callback to monitor several parameters ( I have a multi-task network). Is it possible with just one callback ? Or do I need to do that in many callbacks ??

The ckechpoint creation : 

checkpointer = ModelCheckpoint(filepath='checkpoints/weights-{epoch:02d}.hdf5', monitor='val_O1_categorical_accuracy' , verbose=1, save_best_only=True, mode='max')

The second parameter I want to monitor : val_O2_categorical_accuracy

Doing that in a list will not work. i.e. 

checkpointer = ModelCheckpoint(filepath='checkpoints/weights-{epoch:02d}.hdf5', monitor=['val_O1_categorical_accuracy','val_O2_categorical_accuracy'] , verbose=1, save_best_only=True, mode='max')

TypeError: unhashable type: 'list'

Lilo
  • 640
  • 1
  • 9
  • 22

1 Answers1

2

I am afraid you will have to do it in separate instances. Think about what is happening here - 

checkpointer = ModelCheckpoint(filepath='checkpoints/weights-{epoch:02d}.hdf5', monitor='val_O1_categorical_accuracy' , verbose=1, save_best_only=True, mode='max')

When you are saving a model by monitoring val_O1_categorical_accuracy, here is what it will do in pseudocode - 

for each epoch:
    check the val_O1_categorical_accuracy after updating weights
    if this metric is better in this epoch than the previous ones:
        save the model
    else
        pass

So really specifying multiple monitor is out of scope. In this case it has to be an either/or choice as based on a monitor metric only one model among other conflicting models can be the best one.

Vivek Kalyanarangan
  • 8,951
  • 1
  • 23
  • 42
  • Okay thank you, I think I will add a second metric that computes the mean of the two losses, – Lilo Feb 25 '18 at 07:46
  • yes that's a good call. I have never played with the `monitor` param. May be it does support `callable`. Anyway if your query is resolved please mark as answer so that others searching for something similar will be able to find the help. Otherwise, post the answer yourself in case you have figured it out and mark it – Vivek Kalyanarangan Feb 25 '18 at 07:47