validation loss shows 'no log' during fine-tuning model

Question

I'm finetuning QA models from hugging face pretrained models using huggingface Trainer, during the training process, the validation loss doesn't show. My compute_metrices function returns accuracy and f1 score, which doesn't show in the log as well.

here is my code for trainer set up:

args = TrainingArguments(
    output_dir="./result_albert_nontracking",
    evaluation_strategy="steps",
    save_strategy="epoch",
    max_steps=10000,  
    do_train=True,
    do_eval=True,
    warmup_steps=500,
    num_train_epochs=3,
    weight_decay=0.01,
    learning_rate=5e-5,
    logging_dir='./logs',
    logging_steps=500,
    eval_steps=500,

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=validation_dataset,
    data_collator=data_collator,
    tokenizer=tokenizer,
    compute_metrics=compute_metrics
)
trainer.train()

also have setup logger by:

logger = logging.get_logger(__name__)
logger.setLevel(logging.DEBUG)

here is what I got:

I've tried different checking point ('bert-base-cased','albert-base-v2','roberta-base'), and got the same 'no log'. Any one know what is the problem with that? Thanks in advance!

Answer 1

I encountered a similar issue. The solution was to add return_loss=True argument to my model's forward method.

class CustomMLMModel(PreTrainedModel):
    def forward(self, YOUR_ARGS, return_loss=True):
        ...

That's because transformers Trainer checks (here)1 if a model can_return_loss. As you can see in the source code, this function checks through python's inspection if model's forward method has return_loss as argument and default value is True. See here