1

I'm trying to use this model on huggingface for QA. The code for it is in the link:

from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = "deepset/roberta-base-squad2"

# a) Get predictions
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
    'question': 'Why is model conversion important?',
    'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

# b) Load model & tokenizer
model = AutoModelForQuestionAnswering.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

print(res)
>>>
{'score': 0.2117144614458084,
 'start': 59,
 'end': 84,
 'answer': 'gives freedom to the user'}

However, I can't figure out how to get a loss so I can finetune this model. I was looking at the huggingface tutorial but didn't see anything other than using the Trainer method or another training method in the link (which is not QA):

import torch
from transformers import AdamW, AutoTokenizer, AutoModelForSequenceClassification

# Same as before
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSequenceClassification.from_pretrained(checkpoint)
sequences = [
    "I've been waiting for a HuggingFace course my whole life.",
    "This course is amazing!",
]
batch = tokenizer(sequences, padding=True, truncation=True, return_tensors="pt")

# This is new
batch["labels"] = torch.tensor([1, 1])

optimizer = AdamW(model.parameters())
loss = model(**batch).loss
loss.backward()
optimizer.step()

Say that the true answer is freedom to the user instead of gives freedom to the user

Penguin
  • 1,923
  • 3
  • 21
  • 51

1 Answers1

0

You do not have to get the loss. There is the Trainer class in Hugginface and you can use it to train your model. It is also optimized for hugginface models and contains lots of different kind of deep learning best practices that you might be interested in. See here: https://huggingface.co/docs/transformers/main_classes/trainer

Berkay Berabi
  • 1,933
  • 1
  • 10
  • 26
  • Thanks for the help! I'm familiar with the `Trainer` class, but it's not optimal for my use case. I'm training a meta learning algorithm and need to get a loss for 1 sentence/context at a time. – Penguin Oct 02 '22 at 17:40
  • it would have been better if you stated that in the question. Which model are you using? – Berkay Berabi Oct 02 '22 at 20:13
  • model_name = "deepset/roberta-base-squad2" – Penguin Oct 02 '22 at 20:23