6

What is the use for average_loss? Would someone please give an example of it or explain it in layman's terms?

halfer
  • 19,824
  • 17
  • 99
  • 186
Hossein
  • 24,202
  • 35
  • 119
  • 224

1 Answers1

8

You can check in the caffe.proto file. Line 151 in the present version gives the following comment for average_loss:

Display the loss averaged over the last average_loss iterations
malreddysid
  • 1,342
  • 1
  • 9
  • 12
  • 1
    so what does that mean exactly? what does averaging loss for x iteration do? just averaging and displaying it ? why would I want that? – Hossein Oct 22 '16 at 13:54
  • Instead of the complete average loss. It gives you the loss for the last n iterations. It's just a feature. Might be useful in some networks. – malreddysid Oct 22 '16 at 21:49
  • Thanks, though it would be very desirable to know an intuitive example for it – Hossein Oct 23 '16 at 15:59
  • 8
    Sometimes training is performed with a small batch size and the loss might be noisy. It makes sense to average it over last n iterations to get a better understanding of what is happening. – warmspringwinds Jan 13 '17 at 23:16
  • Thanks a lot, that makes sense – Hossein Jan 27 '17 at 10:43