0

I have a folder with the ruBERT model, which was fine-tuned with the application of the Deeppavlov library. The folder contains the following model files:

enter image description here

How do I convert it to Huggingface format so that I can load it this way?

from transformers import TFAutoModelForSequenceClassification

model_name = "folder_with_ruBERT"
auto_model_rubert = TFAutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name, do_lower_case = False)
senek
  • 45
  • 1
  • 1
  • 9

1 Answers1

1

There is no need to convert the tf checkpoints into huggingface format. The DeepPavlov's pretrained language models are already available as huggingface models.

https://huggingface.co/DeepPavlov

com
  • 2,606
  • 6
  • 29
  • 44
  • It's clear, but how do I load the weights of my finely tuned model, and not the original ruBERT model? – senek Jan 19 '23 at 00:24
  • 1
    @senek you can either fine-tune your model with the newest version of DeepPavlov that's based on huggingface AutoModel, or alternatively you can convert tf checkpoint by using huggingface [script](https://huggingface.co/docs/transformers/converting_tensorflow_models#bert). – com Jan 19 '23 at 07:42
  • It worked, thank you. However, there is a warning Some weights or buffers of the TF 2.0 model TFBertForSequenceClassification were not initialized from the PyTorch model and are newly initialized: ['classifier.weight', 'classifier.bias'] but this is the topic of another question – senek Jan 23 '23 at 05:50