gpt3 - error with the openai api when trying to generate an embedding

Question

I have a python code for create a embedding with openai, but when I try to execute the code, I receive this error:

The server is currently overloaded with other requests. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists.

This is the python code:

# Carga el diccionario de HPOs desde un archivo JSON
with open("hpos.json") as f:
    hpos_dict = json.load(f)

# Crea un diccionario para almacenar los embeddings
hpo_embeddings = {}

i = 0
hposNumber = len(hpos_dict)
# Crea los embeddings y guárdalos en el diccionario
for hpo_id, hpo_descs in hpos_dict.items():
    embedding_list = []
    for hpo_desc in hpo_descs:
        response = openai.Embedding.create(
            input=hpo_desc,
            model="text-embedding-ada-002"
        )
        embedding_list.append(response["data"][0]["embedding"])
    hpo_embeddings[hpo_id] = embedding_list
    i = i + 1
    print( str(i) + "/" + str(hposNumber) )

# Guarda el diccionario de embeddings en un archivo JSON
with open("hpo_embeddings.json", "w") as f:
    json.dump(hpo_embeddings, f)

Please remember that Stack Overflow is not your favourite Python forum, but rather a question and answer site for all programming related questions. Thus, always include the tag of the language you are programming in, that way other users familiar with that language can more easily find your question. Take the [tour] and read up on [ask] to get more information on how this site works, then [edit] the question with the relevant tags. — Adriaan, Mar 14 '23 at 10:17

score 0 · Answer 1 · edited Jul 27 '23 at 13:53

One way to resolve this issue is to wait and retry your request at a later time when the server load has decreased. Another option is to reduce the number of requests you're making to the OpenAI API to avoid overwhelming the server. You can also consider using a different API endpoint or model that may be less busy as an alternative solution.

Or you can keep retrying to call the API till you receive the response, checkout here's an updated version of your code to achieve this:

import time
import json
import openai

# Set up OpenAI API credentials
openai.api_key = "YOUR_API_KEY"

# Load the HPOs dictionary from a JSON file
with open("hpos.json") as f:
    hpos_dict = json.load(f)

# Create a dictionary to store the embeddings
hpo_embeddings = {}

# Loop through each HPO and create embeddings
for hpo_id, hpo_descs in hpos_dict.items():
    embedding_list = []
    for hpo_desc in hpo_descs:
        # Retry the API call if it fails due to server overload
        while True:
            try:
                response = openai.Embedding.create(
                    input=hpo_desc,
                    model="text-embedding-ada-002"
                )
                embedding_list.append(response["data"][0]["embedding"])
                break # Exit the loop if the API call succeeds
            except openai.error.APIError as e:
                # If the API call fails, wait and retry after a delay
                print("API error:", e)
                print("Retrying in 10 seconds...")
                time.sleep(10)
        hpo_embeddings[hpo_id] = embedding_list

    print(f"Processed HPO {hpo_id}, {len(hpo_descs)} descriptions")

# Save the embeddings to a JSON file
with open("hpo_embeddings.json", "w") as f:
    json.dump(hpo_embeddings, f)

This code help me! Thank you so much, I just had to change the "except openai.error.APIError" exception to "except openai.error.RateLimitError" and it worked. — Alberto, Mar 16 '23 at 09:46

gpt3 - error with the openai api when trying to generate an embedding

1 Answers1