0

Can I use roberta for tokenizer while bert for model?

from transformers import RobertaTokenizerFast

tokenizer = RobertaTokenizerFast.from_pretrained("./bert_tokenizer", max_len=512)

from transformers import BertForMaskedLM
config = BertConfig()
bert= BertForMaskedLM(config)
duoduolikes
  • 33
  • 5

1 Answers1

1

You cannot use RoBERTa tokenizer for BERT model. The reason is that the vocabulary for BERT and RoBERTa are different. Thus a word-piece token which is present in RoBERTa's vocabulary may not be present in BERT's vocabulary.

Ashwin Geet D'Sa
  • 6,346
  • 2
  • 31
  • 59