1

I use AWS SageMaker to run a training with AllenNLP. In order to track the loss and metrics I need to have them printed on the log INFO level during training (or at least after each epoch). However, when I run the training all loss and metric information is printed to the console without using the logger:

2021-07-12 16:39:25,799 - INFO - allennlp.training.trainer - Epoch 2/24
2021-07-12 16:39:25,803 - INFO - allennlp.training.trainer - Worker 0 memory usage: 1.5G
2021-07-12 16:39:25,806 - INFO - allennlp.training.trainer - Training
accuracy: 0.1116, batch_loss: 0.4598, loss: 0.4742 ||: 100%|##########| 8/8 [00:13<00:00,  1.64s/it]
2021-07-12 16:39:40,229 - INFO - allennlp.training.trainer - Validating
accuracy: 0.2000, batch_loss: 0.4377, loss: 0.4215 ||: 100%|##########| 2/2 [00:03<00:00,  1.87s/it]

So far, I could not find anything in the issues or on StackOverflow. As I said, having the loss and metrics logged on INFO level once per epoch would be totally fine.

Also in this example it seems like the loss and metrics are logged the way I would like to have it.

Richard
  • 514
  • 3
  • 9
  • 1
    If you use `ConsoleLoggerCallback` in your trainer (this can be done by either explicitly specifying the callback in your config, or setting `enable_default_callbacks` to `True`), that should log the metrics at INFO level once per epoch. – akshitab Jul 16 '21 at 04:05
  • Thanks! I will give it a try. – Richard Jul 20 '21 at 13:49
  • 1
    Actually, it was a version issue. I was using `2.1.0` for some docker reason. With the update on `2.6.0` it prints as expected by default. Thank you! – Richard Jul 20 '21 at 14:14

0 Answers0