GPT-3 long input posts for Question Answering

Question

From my understanding, GPT-3 is "trained" for a specific task by including some labelled examples before the desired/test example. In Question Answering, this includes a context and a question. In this situation, the input prompt can become long. How do people address this?

I am using the Hugging Face GPT-J implementation, and there is an input token limit (of 2000). However, when including multiple qa examples in the prompt (especially with the contexts), it quickly reaches this limit, limitting the amount of example prompts to be inputted. Does anyone know how this issue is handled in a GPT-J setting, especially for QA?

Could you give us examples or the exact qa prompt entry that you saw? My first reaction is to crop the context — Mehdi, Feb 08 '22 at 04:18

score 0 · Answer 1 · answered Mar 24 '22 at 13:08

0

Unfortunately GPT-3 and GPT-J both have a 2048 token context limitation, and there's nothing you can do about it.

On my NLP Cloud API, the solution I suggest in general is to fine-tune GPT-J. Fine-tuning GPT-J is like giving ton of context to the model.

answered Mar 24 '22 at 13:08

Julien Salinas

1,059
1
10
23

There is now a GPT-3 model (`gpt-3.5-turbo-16k`) that handles up to 16k tokens – medecau Jul 04 '23 at 11:38

GPT-3 long input posts for Question Answering

1 Answers1