1

Good morning!

I just want to clarify - is the batch_size parameter in model.fit declaring how many samples go in, or is the number of samples that go in at a time x/batch_size, where there are a total of x samples?

That is, suppose we have 20,000 samples and we make a batch size of 100. Does that mean 200 samples are passed in at a time (meaning 100 batches), or 100 samples?

I ask because https://deeplizard.com/learn/video/Skc8nqJirJg says "If we passed our entire training set to the model at once (batch_size=1), then the process we just went over for calculating the loss will occur at the end of each epoch during training", implying that it's one batch. However, batch_size seems to mean something difference based on its name, so I wanted to clarify.

Thank you!

Note: there is another question like this, but it wasn't answered - How BatchSize in Keras works ? LSTM-WithState-Time Series

That adds: how are those samples chosen?

Janakored
  • 15
  • 4

1 Answers1

2

From TensorFlow documentation:

batch_size: Integer or None. Number of samples per gradient update. If unspecified, batch_size will default to 32. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

So it's the number of samples used before a gradient update. If batch_size is equal to 1, then there will be one gradient update for each sample (and therefore, num_samples for each epoch)

For instance, for the example you cited: if we have 20,000 samples and we make a batch size of 100, 100 samples are passed at a time.

That adds: how are those samples chosen?

It depends whether the shuffle argument of the fit method is True or not. If it is, they are taken randomly until all the samples have been selected (end of epoch). If it's not, they are taken sequentially

jackve
  • 309
  • 1
  • 11
  • Also note, from your example if you had 20050 samples in your training set and a batch size of 100 you would have 201 batches total per epoch. The first 200 batches would have 100 samples per batch and the last batch would only have 50 samples. This isn't necessarily an issue for the majority of models or data sets though if you had 2001 samples and a batch size of 100 your last gradient update would depend on a single data sample and provide the network with "uneven"/unfair gradient information from whatever the last sample may be. – KDecker Jul 02 '21 at 13:31