3

For my use case, I need to obtain the logits from T5's forward() method without inputting labels. I know that forward() and .generate() are different (see here). I have also seen this post in which the logits were obtained but labels had to be generated first. Is it possible to obtain the logits from the forward() method without inputting the labels?

muhleeshe
  • 61
  • 4

1 Answers1

0

Yes, you can do it. Huggingface library does not requires labels strictly as input to the forward method. See for example here for Bert: https://github.com/huggingface/transformers/blob/v4.21.1/src/transformers/models/bert/modeling_bert.py#L1079

Labels is an optional parameter. Also, later in the forward method, they check if labels are None or not. If they are None, they do not compute the loss and just return the output.

See here: https://github.com/huggingface/transformers/blob/v4.21.1/src/transformers/models/bert/modeling_bert.py#L1135

And the forward returns a dictionary that also contains logits.

https://github.com/huggingface/transformers/blob/v4.21.1/src/transformers/models/bert/modeling_bert.py#L1147

But please do look at the specific model's code you are using.

Berkay Berabi
  • 1,933
  • 1
  • 10
  • 26
  • Hi Berkay, thank you so much for the reply! It looks like I do not exactly need the labels to run a forward pass through the model. Correct me if I am wrong but it does seem like I do need the decoder input ids to run forward pass through the T5 model, which I have only seen being obtained using the labels and tokenizer. Do you know if there is a way to get the decoder input ids without using the labels? – muhleeshe Aug 24 '22 at 00:40
  • no you donot need the decoder ids. Bert does not even have a decoder part. Just try to run the code that I shared and refer to documentation. It must work. – Berkay Berabi Aug 24 '22 at 11:31
  • I have tried but I get this error: "You have to specify decoder_input_ids or decoder_input_embeds". Note: I am using T5 not BERT – muhleeshe Aug 24 '22 at 20:43