0

I am doing an experiment where I need to keep track of when the model would eventually early stop, but without actually early stopping training. Why? Because I need to analyze how the model would actually behave after starting to apparently overfit.

So, is there a way of keeping track of when the model would early stop without actually early stopping?

Of course, one could manually keep track of the loss (or performance) and manually decide when the model would be overfitting (e.g. after 5 iterations of no improvement on the validation data, we could record a possible "overfitting" situation), but I was wondering if there's a callback or something that would allow me to do this automatically.

nbro
  • 15,395
  • 32
  • 113
  • 196
  • What do you exactly want? Do you want to it just store the epoch at which it would do early-stopping but then continue training? i.e. Are you just interested in the epoch number? – today Apr 14 '20 at 18:17
  • @today Well, yes, I am interested in the information that would allow me to investigate at each epoch the model started to overfitting. As I say in the question above, this can be done manually, i.e. I can create a callback that writes to a file the epoch number, but I was wondering if there's a built-in callback that already does this or something like that. Anyway, if you want to provide the source code for a manually written callback, feel free! Otherwise, I will provide it below later. – nbro Apr 14 '20 at 18:20
  • I am not aware of any **built-in** callback that does such a job (i.e. recognizing earlystopping but not stopping the training). There is only the default `History` callback which store all the loss and accuracy information during training of the model; but as I said it does not recognize earlystopping by itself, rather you need to process its information (e.g. using a function) after the training is finished to find the desired epoch and other desired information. – today Apr 14 '20 at 18:37
  • @today Yeah, I guess I will use that solution, as I was thinking. Thank you anyway! – nbro Apr 14 '20 at 19:32

0 Answers0