Does tf.nn.softmax_cross_entropy_with_logits account for batch size?
In my LSTM network, I feed batches of different sizes, and I would like to know whether or not I should normalize error with respect to batch size before optimizing.
Asked
Active
Viewed 1,261 times
1
nbro
- 15,395
- 32
- 113
- 196
casparjespersen
- 3,460
- 5
- 38
- 63
-
1I don't thing the batch size has anything to do with it. the lose is per example (however you define example). you can reduce mean afterwards to get the average loss for the batch – user2717954 Jun 05 '16 at 05:42
1 Answers
0
In the documentation it says that the softmax_cross_entropy_with_logits returns a vector with length equal to the batch size. To get a scalar cost you can do tf.reduce_mean on that vector. Then your loss will not be affected by the batch size.
