I am trying to learn to finetune GPT models, and am most familiar with working with Python using IDEs (eg Spyder). Would anyone know why the following code in my IDE (Spyder) gives no response, no tracebacks, no errors? I do not see any sign of the model appearing in the OpenAI Playground too.
Some notes: Test1 is the nickname for my model. API key is removed in the code. I know i'm not supposed to code API keys as string - will eventually cloak it. I'm trying to use the ada model (cheapest, just for testing). My prompts/answers are in excel, and the code successfully saves it as a csv with prompt/response pairs. I get "FileNotFoundError: [WinError 2] The system cannot find the file specified" if i remove "shell=True".
import pandas as pd
import openai
import subprocess
openai.api_key = 'xxxx'
datafile = "C:/xx/xx/xx.xlsx"
df=pd.read_excel(datafile,sheet_name="Sheet1",index_col = 0,skiprows=range(2),usecols = "A:D",nrows=300,parse_dates=True)
# df = pd.read_csv("out_openai_completion.csv")
prepared_data = df.loc[:,['Trigger','Reply']]
prepared_data.rename(columns={'sub_prompt':'prompt', 'response_txt':'completion'}, inplace=True)
prepared_data.to_csv('prepared_data.csv',index=False)
# prepared_data.csv --> prepared_data_prepared.json
subprocess.run('openai tools fine_tunes.prepare_data --file prepared_data.csv --quiet'.split(),shell=True)
# ## Start fine-tuning
subprocess.run('openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model text-davinci-002 --suffix "Test1"'.split(),shell=True)
subprocess.run('openai api fine_tunes.get -i <SRC_Test1>',shell=True) #to check progress/status
