I have tried fine tuning a GPT-3 model for the purpose of text classification to classify whether two names match such as 'William Jonathan' and 'William J' and the label would be yes/no, yes indicating that two names are matching and no indicating that they aren't. I have created a large number of examples related to different scenarios such as names being spelled differently, abbreviations, missing token, etc. After fine-tuning the model on GPT-3 using examples that look like this with a jsonl format:
{"text": "Are the following two names the same?\nWilliam Jonathan\nWilliam J", "label": "Yes"}
It is not able to do binary classification, however it outputs a large number of labels next to each other, for instance:
Prompt: Are the following two names the same?\nWilliam Jonathan \nWilliam J
Completion: YesYesNoYesNoYesYesNoYesNoYesNoYesNoYesYes
Any idea on how I can perform binary text classification using GPT-3 on an example similar to the above?
