2

I'm building a Keras sequential model to do a binary image classification. Now when I use like 70 to 80 epochs I start getting good validation accuracy (81%). But I was told that this is a very big number to be used for epochs which would affect the performance of the network.

My question is: is there a limited number of epochs that I shouldn't exceed, note that I have 2000 training images and 800 validation images.

Lior Magen
  • 1,533
  • 2
  • 15
  • 33
rana hd
  • 355
  • 3
  • 18
  • [While not an exact duplicate, this method is one I have used in the past for selecting an how many epochs to use](https://stackoverflow.com/questions/47421951/keras-optimal-epoch-selection) – Edeki Okoh Jun 26 '19 at 22:37
  • No, there isn't a max number of epochs that you should never exeed. This also depends on other parameters such as the learning rate. I'd like to refer you to [this answer](https://datascience.stackexchange.com/a/53036/34269) which shares the same context with yours. – Djib2011 Jun 26 '19 at 23:11

3 Answers3

4

If the number of epochs are very high, your model may overfit and your training accuracy will reach 100%. In that approach you plot the error rate on training and validation data. The horizontal axis is the number of epochs and the vertical axis is the error rate. You should stop training when the error rate of validation data is minimum.

You need to have a trade-off between your regularization parameters. Major problem in Deep Learning is overfitting model. Various regularization techniques are used,as

i) Reducing batch-size

ii) Data Augmentation(only if your data is not diverse)

iii) Batch Normalization

iv) Reducing complexity in architecture(mainly convolutional layers)

v) Introducing dropout layer(only if you are using any dense layer)

vi) Reduced learning rate.

vii) Transfer learning

Batch-size vs epoch tradeoff is quite important. Also it is dependent on your data and varies from application to application. In that case, you have to play with your data a little bit to know the exact figure. Normally a batch size of 32 medium size images requires 10 epochs for good feature extraction from the convolutional layers. Again, it is relative

Hamsa
  • 476
  • 5
  • 23
  • why using dropout only for dense layer? – Khan Feb 11 '20 at 19:02
  • batch normalization is generally used between two convolutional layers. Dropout is less effective in regularizing convolutional layers. – Hamsa Feb 11 '20 at 19:23
2

There's this Early Stopping function that Keras supply which you simply define.

EarlyStopping(patience=self.patience, verbose=self.verbose, monitor=self.monitor)

Let's say that the epochs parameter equals to 80, like you said before. When you use the EarlyStopping function the number of epochs becomes the maximum number of epochs.

You can define the EarlyStopping function to monitor the validation loss, for example, when ever this loss does not improve no more it'll give it a few last chances (the number you put in the patience parameter) and if after those last chances the monitored value didn't improve the training process will stop.

The best practice, in my opinion, is to use both EarlyStopping and ModelCheckpoint, which is another callback function supplied in Keras' API that simply saves your last best model (you decide what best means, best loss or other value that you test your results with).

This is the Keras solution for the problem your trying to deal with. In addition there is a lot of online material that you can read about how to deal with overfit.

Lior Magen
  • 1,533
  • 2
  • 15
  • 33
1

Yaa! Their is a solution for your problem. Select epochs e.g 1k ,2k just use early stoping on your neural net.

Early Stopping: Keras supports the early stopping of training via a callback called Early-stopping.

This callback allows you to specify the performance measure to monitor, the trigger, and once triggered, it will stop the training process. For example you apply a trigger that stop the training if accuracy is not increasing in previous 5 epochs. So keras will see previous 5 epochs through call backs and stop training if your accuracy is not increasing

Early Stopping link :

Sohaib Anwaar
  • 1,517
  • 1
  • 12
  • 29