2

I'm running the MNIST Neural Network example and when training the classifier, the Validation Loss column (and training loss) has some values highlighted in green. If i have the learning rate at 0.01, all are green, however if i increase it to 0.1, only half of them are highlighted.

What does this highlighting mean if anything?

Output example:

Validation Loss Highlighting

Waddas
  • 1,353
  • 2
  • 16
  • 28

1 Answers1

1

Without knowing the internals, i see a pattern there.

  • Color each row as long as there is an improvement (minimization -> it's lower)
    • train-loss blue
    • val-loss green
  • Stop coloring when loss / validation did increase
    • Don't start coloring again after this one-time increase

Different learning-rates will effect in different train-loss / val-loss paths and therefore some possible different coloring.

Your example fits well there, as a lower learning-rate is usually more stable (from iteration to iteration) in regards to lowering the loss (more blue, more green).

sascha
  • 32,238
  • 6
  • 68
  • 110