Bert using transformer's pipeline and encode_plus function

Question

when I use:

modelname = 'deepset/bert-base-cased-squad2'
model = BertForQuestionAnswering.from_pretrained(modelname)
tokenizer = AutoTokenizer.from_pretrained(modelname)
nlp = pipeline('question-answering', model=model, tokenizer=tokenizer)

result = nlp({'question': question,'context': context})

it doesn't crash. However when i use encode_plus():

modelname = 'deepset/bert-base-cased-squad2'
model = BertForQuestionAnswering.from_pretrained(modelname)
tokenizer = AutoTokenizer.from_pretrained(modelname)

inputs= tokenizer.encode_plus(question,context,return_tensors='pt')

I have this error:
The size of tensor a (629) must match the size of tensor b (512) at non-singleton dimension 1

which I understand but why I don't have the same error in the first case? Can someone explain the difference?

Can you please provide a minimal-reproducible example. https://stackoverflow.com/help/minimal-reproducible-example — Ashwin Geet D'Sa, Jul 22 '21 at 10:02

score 0 · Answer 1 · answered Aug 12 '21 at 06:23

The reason for getting an error in the second code is that the input data does not fit in the pytorch tensor. For this you have to set truncation flag as True when calling the tokenizer. Thus, when data that will not fit in the tensor arrives, it only takes as much as it fits. i.e:

tokenizer = AutoTokenizer.from_pretrained('deepset/bert-base-cased-squad2',truncation= True )

There is no problem when using the pipeline, probably because the developers of the pre-trained model used in the pipeline apply this process by default.

Bert using transformer's pipeline and encode_plus function

1 Answers1