1

I have fine tuned a custom dataset using GPT3. I created a simple program to take user input (a question) and return the correct response. The program works, however it returns additional question and answers from the dataset I uploaded to the model.

I tried to reduce the max tokens cap and have set the temperature to 0, but I cannot seem to figure out how to stop the program from returning the additional questions and answers. Has anyone encountered this problem and if so how can I fix it?

Here is my code:

import openai

openai.api_key = "MY_API_KEY"

def respond(prompt):
    completions = openai.Completion.create(
        engine="MY_FINED_TUNED_MODEL",
        prompt=prompt,
        max_tokens=50,
        n=1,
        stop=None,
        temperature=0,
    )

    message = completions.choices[0].text
    return message

while True:
    prompt = input("Enter your question: ")
    if prompt.lower() == "end":
        break
    response = respond(prompt)
    print(response)
devpolo
  • 2,487
  • 3
  • 12
  • 28
Archie
  • 13
  • 3

1 Answers1

1

You can handle this by providing more explicit details to the prompt and even providing one-shot or many-shot examples in the prompt.

For example say this was a Q&A bot for a computer store:

question = 'Do you sell computers?'

prompt = f"""Act as a question and answer AI. You will be provided with a question and you should respond based on your fine-tuning data. Provide the most appropriate answer to the question. Provide a single answer. Do not provide additional questions and answers.

Example question:
Do you sell cars?

Example response:
Sorry, we are a computer retailer and only sell computers.

Question:
{question}

Response:"""

Depending on your responses from the model you can also use a stop sequence. For example using \r\n will cause the model to stop responding when it generates a new line.

If you provide more details of the types of questions and responses you are getting from the model I can provide you a better prompt.

Kane Hooper
  • 1,531
  • 1
  • 9
  • 21
  • I have set n=1 to generate just 1 response/completion so I was expecting it to react accordingly. In your recommendation, would the user have to submit that prompt for each question? Or would I add that prompt into the code, and if so then how would I do that? The questions are regarding operational guidelines - so a user could ask a question such as: "Can a person be employed as a volunteer?" and the response would be "No, a person cannot be employed as a volunteer." I am getting the correct responses but I would like to eliminate the additional questions and answers it is returning. – Archie Feb 10 '23 at 15:14
  • The thing to remember is the GPT is just a very sophisticated autocomplete engine. If you care getting repeat questions and answers, you may need to look at your fine-tuning. But from a prompt perspective, you would handle the prompt engineering in your code. My example above shows how you can programmatically craft the code and inject the users question into the prompt. What you are running into can be handled by crafting the prompt in your code and injecting the user question. – Kane Hooper Feb 11 '23 at 05:59