0

After installing Neptune.ai for occasional ML experiments logging, it became included by default into the list of callbacks in all transformers.trainer runs. As a result, it requires proper initialisation with token or else throws NeptuneMissingConfiguration error, demanding token and project name. This is really annoying, I'd prefer Neptune callback to limit itself to warning or just have it disabled if no token is provided. Unfortunately there is no obvious way to disable this callback, short of uninstalling Neptune.ai altogether. The doc page at https://huggingface.co/docs/transformers/main_classes/callback states that this callback is enabled by default and gives no way to disable it (unlike some other callbacks that can be disabled by environment variable).

Question: how to disable Neptune callback on per run basis?

Poe Dator
  • 4,535
  • 2
  • 14
  • 35

2 Answers2

1

Apparently this piece of code after trainer initialization helps:

for cb in trainer.callback_handler.callbacks:
    if isinstance(cb, transformers.integrations.NeptuneCallback):
        trainer.callback_handler.remove_callback(cb)

Still it would be good if Transformers or Neptune team provided more flexibility with this callback.

Poe Dator
  • 4,535
  • 2
  • 14
  • 35
0

To disable Neptune callback in transformers trainer runs, you can pass the --no-neptune flag to the trainer.train() function.

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    no_neptune=True
)
trainer.train()
Ignacio Horcada Bernal
  • 1
  • 2
  • this would be a nice solution, but it does not work. there is no such argument in trainer. Also nothing like this in docs. transformers version='4.24.0' – Poe Dator Dec 04 '22 at 17:26