Unexpected keyword argument 'unk_token'

Asked Sep 08 '22 at 15:03

Active Sep 08 '22 at 15:03

Viewed 125 times

When trying to load this tokenizer I am getting this error but I don't know why it can't take the ink_token strangely. Any ideas?

tokenizer = tokenizers.SentencePieceUnigramTokenizer(unk_token="", eos_token="", pad_token="")

----> 1 tokenizer = tokenizers.SentencePieceUnigramTokenizer(unk_token="", eos_token="", pad_token="")

TypeError: init() got an unexpected keyword argument 'unk_token'

asked Sep 08 '22 at 15:03

Antoine23

0 Answers0