Trying to train.py a new language on donut based on a corpora I generated with their synthDOG tool, running the command python train.py --config config/base.yaml --exp_version "base" on up-to-date Windows 11 inside a conda virtualenv. Dev mode in Windows is activated, and I've launched Anaconda as admin, cmd also shows Administrator: at the top.
The error is as follows:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\envs\Donut\lib\site-packages\transformers\tokenization_utils_base.py", line 1958, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "C:\ProgramData\Anaconda3\envs\Donut\lib\site-packages\transformers\models\xlm_roberta\tokenization_xlm_roberta.py", line 168, in __init__
self.sp_model.Load(str(vocab_file))
File "C:\ProgramData\Anaconda3\envs\Donut\lib\site-packages\sentencepiece\__init__.py", line 905, in Load
return self.LoadFromFile(model_file)
File "C:\ProgramData\Anaconda3\envs\Donut\lib\site-packages\sentencepiece\__init__.py", line 310, in LoadFromFile
return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
OSError: Not found: "C:\Users\me\.cache\models--naver-clova-ix--donut-base\snapshots\a959cf33c20e09215873e338299c900f57047c61\sentencepiece.bpe.model": No such file or directory Error #2
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\me\Documents\Kontron\donut-master\train.py", line 149, in <module>
train(config)
File "C:\Users\me\Documents\Kontron\donut-master\train.py", line 57, in train
model_module = DonutModelPLModule(config)
File "C:\Users\me\Documents\Kontron\donut-master\lightning_module.py", line 30, in __init__
self.model = DonutModel.from_pretrained(
File "C:\Users\me\Documents\Kontron\donut-master\donut\model.py", line 594, in from_pretrained
model = super(DonutModel, cls).from_pretrained(pretrained_model_name_or_path, revision="official", *model_args, **kwargs)
File "C:\ProgramData\Anaconda3\envs\Donut\lib\site-packages\transformers\modeling_utils.py", line 2498, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "C:\Users\me\Documents\Kontron\donut-master\donut\model.py", line 390, in __init__
self.decoder = BARTDecoder(
File "C:\Users\me\Documents\Kontron\donut-master\donut\model.py", line 159, in __init__
self.tokenizer = XLMRobertaTokenizer.from_pretrained(
File "C:\ProgramData\Anaconda3\envs\Donut\lib\site-packages\transformers\tokenization_utils_base.py", line 1804, in from_pretrained
return cls._from_pretrained(
File "C:\ProgramData\Anaconda3\envs\Donut\lib\site-packages\transformers\tokenization_utils_base.py", line 1960, in _from_pretrained
raise OSError(
OSError: Unable to load vocabulary from file. Please check that the provided vocabulary is accessible and not corrupted.
When running by default, the cache link is incorrect, as it generates something like "C:\User\me/.cache\etc.". I manually changed the cache, but still after downloading about a GB of model files, the folder only contains SYMLINK files that are 0 KB. Even when I manually downloads all the files and add them in, the path won't be recognized (please do note the path is now NOT mixing / and \ together like it did by default). Copy-pasting the apparently erroneous file path from the cmd stack trace to a python with open() command, it seems to open it just fine. I have no idea what's wrong, but I'd really like some help, I'm going crazy. It's saying files aren't there, but they are.
