20

I'm using Keras to predict a time series. As standard I'm using 20 epochs. I want to check if my model is learning well, by predicting for each one of the 20 epochs.

By using model.predict() I'm getting only one prediction among all epochs (not sure how Keras selects it). I want all predictions, or at least the 10 best.

Would anyone know how to help me?

Scott
  • 4,974
  • 6
  • 35
  • 62
aabujamra
  • 4,494
  • 13
  • 51
  • 101

3 Answers3

16

I think there is a bit of a confusion here.

An epoch is only used while training the neural network, so when training stops (in this case, after the 20th epoch), then the weights correspond to the ones computed on the last epoch.

Keras prints current loss values on the validation set during training after each epoch. If the weights after each epoch are not saved, then they are lost. You can save weights for each epoch with the ModelCheckpoint callback, and then load them back with load_weights on your model.

You can compute your predictions after each training epoch by implementing an appropriate callback by subclassing Callback and calling predict on the model inside the on_epoch_end function.

Then to use it, you instantiate your callback, make a list and use it as keyword argument callbacks to model.fit.

Dr. Snoopy
  • 55,122
  • 7
  • 121
  • 140
  • Hello Matias! If you could have a look on this one it would be highly appreciated: http://stackoverflow.com/questions/36895627/python-keras-creating-a-callback-with-one-prediction-for-each-epoch – aabujamra Apr 27 '16 at 20:06
11

The following code will do the desired job:

import tensorflow as tf
import keras

# define your custom callback for prediction
class PredictionCallback(tf.keras.callbacks.Callback):    
  def on_epoch_end(self, epoch, logs={}):
    y_pred = self.model.predict(self.validation_data[0])
    print('prediction: {} at epoch: {}'.format(y_pred, epoch))

# ...

# register the callback before training starts
model.fit(X_train, y_train, batch_size=32, epochs=25, 
          validation_data=(X_valid, y_valid), 
          callbacks=[PredictionCallback()])
Sandipan Dey
  • 21,482
  • 2
  • 51
  • 63
  • 4
    this seems causing memory leak – York Yang Apr 27 '20 at 10:19
  • it does not work in TF 2.x: AttributeError: 'PredictionCallback' object has no attribute 'validation_data' – Fred Sousa May 12 '21 at 17:22
  • This callback structure will call predict again, even though keras has already made predictions during the epoch, doubling the prediction workload. Is it not possible to access the predictions we know keras has already made during the epoch? – tim654321 Nov 24 '22 at 01:16
3

In case you want to make predictions on the test data, after every epoch while the training is going-on you can try this

class CustomCallback(keras.callbacks.Callback):
    def __init__(self, model, x_test, y_test):
        self.model = model
        self.x_test = x_test
        self.y_test = y_test

    def on_epoch_end(self, epoch, logs={}):
        y_pred = self.model.predict(self.x_test, self.y_test)
        print('y predicted: ', y_pred)

You need mention the callback during model.fit

model.sequence()
# your model architecture
model.fit(x_train, y_train, epochs=10, 
          callbacks=[CustomCallback(model, x_test, y_test)])

Similar to on_epoch_end there are many other methods provided by keras

on_train_begin, on_train_end, on_epoch_begin, on_epoch_end, on_test_begin,
on_test_end, on_predict_begin, on_predict_end, on_train_batch_begin, on_train_batch_end,
on_test_batch_begin, on_test_batch_end, on_predict_batch_begin,on_predict_batch_end
Harshal Deore
  • 1,050
  • 1
  • 11
  • 11
  • Similar to the other answer above, this callback structure will call predict again, even though keras has already made predictions during the epoch, doubling the prediction workload. Is it not possible to access the predictions we know keras has already made during the epoch? – tim654321 Nov 24 '22 at 01:17