0

I am trying to run an experiment with GPT2; i.e., I use model = GPT2Model.from_pretrained('gpt2-xl')

The error I get is a traceback which leads to OSError: [Errno 28] No space left on device: '/home/username/.cache/huggingface/transformers/tmpvuvw8j0t'.

This is unlikely to be an error with my code itself, because the exact same code works fine for GPT3-1.3B and T5-Large (except of course for the model= .... line).

I think the issue is that it tries to download the GPT2 model and runs out of space on the device. The /home/username directory has a pretty small amount of storage; the /data/username directory is where most of the storage is. I'm not quite sure how to redirect it to download the weights on the latter directory, or if that would even help.

I'd really appreciate any help in resolving this!

nlp4892
  • 61
  • 7

2 Answers2

1

I managed to figure out how to solve it without having access to root privileges. Typing export TRANSFORMERS_CACHE=/directory solved the issue.

nlp4892
  • 61
  • 7
0

As a workaround, you could bind-mount a directory from a file system with more space available:

# mount --bind /data/username/huggingface-cache /home/username/.cache/huggingface

This mount exists until you reboot your system. If you want to make it persistent, you can add it to your /etc/fstab file to be mounted at boot.

knittl
  • 246,190
  • 53
  • 318
  • 364
  • Thanks for the comment. Unfortunately, it gives me "only root can use "--bind" option"; I don't have root privileges on this server. – nlp4892 Jan 09 '23 at 21:19