Keras callback for model class API

Question

I'd like to compute AUROC (Area Under ROC Curve) for every epoch and store them for later display. My model is Model API.

Is it possible to create a similar snippet as follow, which uses callbacks in case of Sequential API? Knowing that self.model, self.validation_data and self.model.predict are not callable for Model API.

import keras
from sklearn.metrics
import roc_auc_score

class Histories(keras.callbacks.Callback):

def on_train_begin(self, logs={}):
    self.aucs = []
    self.losses = []

def on_train_end(self, logs={}):
    return

def on_epoch_begin(self, epoch, logs={}):
    return

def on_epoch_end(self, epoch, logs={}):
    self.losses.append(logs.get('loss'))
    y_pred = self.model.predict(self.model.validation_data[0])
    self.aucs.append(roc_auc_score(self.model.validation_data[1], y_pred))
    return

def on_batch_begin(self, batch, logs={}):
    return

def on_batch_end(self, batch, logs={}):
    return

Here is my main:

histories = my_callbacks.Histories()
history = model.fit_generator(train_gen.generate(labels, partition['train']),
                        steps_per_epoch= len(partition['train'])//args.batch_size,
                        epochs=args.nb_epochs,
                        verbose=1,
                        callbacks = [histories],
                        validation_data = test_gen.generate(labels, partition['test']),
                        validation_steps = len(partition['test'])//args.batch_size,
                        use_multiprocessing=True);

Will the AUC be computed for every batch generated by test_gen (test data generator)? What should I do to compute AUCs on each test batch then aggregate them at the end of each epoch?

Thank you for your help.

Keras callback for model class API

0 Answers0