1

What is the difference between losses/clone_0/softmax_cross_entropy_loss and losses/clone_0/aux_loss/value in inception-v4? Currently, I'm training a large-scale model using tf-slim and inception-v4 network on 4 GPUs(--num_clones=4 ). but these two charts are completely different. after 190K steps with batch-size=128, I get these charts:Losses

as you can see in the image total loss and have a similar trend. but the softmax_cross_entropy have a completely different procedure! Which one of these losses can describe the training procedure better?

R.y.z
  • 213
  • 2
  • 15

1 Answers1

0

You should use the first, main one. You can read about the auxiliary head and its loss in here Does the Inception Model have two softmax outputs?

This aux_loss is defined here: https://github.com/tensorflow/models/blob/4bd29ac0ba1004d7393b7d029b05257dffd5cbe6/inception/inception/inception_model.py#L135

iga
  • 3,571
  • 1
  • 12
  • 22