Refining the Code: Python Script for API and Creating a new column from it

Question

Hey guys so this is gonna be a tall order but I need help refining this code that still doesn't do exactly what I want it to do. I am a research student trying to utilize the Edamam Nutrition Analysis API to get the calorie info for foods from this dataset.

I need help refining this code, for it connects to the API and gets the food info, but here are my concerns:

It doesn't cycle through the api keys in the keys list every 10 query calls like I want it to. The key value within the headers dictionary is supposed to change to the next API key in the keys list, as seen from this sample output:

Detected Word:  waffle
Food Stats: 
 {"message":"You have exceeded the DAILY quota for requests on your current plan, BASIC. Upgrade your plan at https:\/\/rapidapi.com\/edamam\/api\/edamam-nutrition-analysis"}
Out of calls, restarting in 30 secs...
x at 1
Key 1 : 16d7ecf46dmshc432a0e5b44478bp18c531jsn775dccbe9451
Detected Word:  butter
Food Stats: 
 {"message":"You have exceeded the DAILY quota for requests on your current plan, BASIC. Upgrade your plan at https:\/\/rapidapi.com\/edamam\/api\/edamam-nutrition-analysis"}
Out of calls, restarting in 30 secs...
x at 2 
Key 2 : 16d7ecf46dmshc432a0e5b44478bp18c531jsn775dccbe9451    # Supposed to
# switch to next key in keys list upon this error message happening
Detected Word:  syrup
Food Stats: 
 {"message":"You have exceeded the DAILY quota for requests on your current plan, BASIC. Upgrade your plan at https:\/\/rapidapi.com\/edamam\/api\/edamam-nutrition-analysis"}
Out of calls, restarting in 30 secs...
x at 3 # AFTER 2 IT IS SUPPOSED TO RESET TO 0!!
Key 3 : 16d7ecf46dmshc432a0e5b44478bp18c531jsn775dccbe9451
Detected Word:  ham
Food Stats: 
 {"message":"You have exceeded the DAILY quota for requests on your current plan, BASIC. Upgrade your plan at https:\/\/rapidapi.com\/edamam\/api\/edamam-nutrition-analysis"}
Out of calls, restarting in 30 secs...
x at 4

I think I am using too many if and else statements, and I did this recently because I had some if statements without else and thought adding the else would ensure that all the code gets executed.
Is my programming logic sound?
Does anybody know of a food api that does what the Edamam Nutrition API does but is FREE? I am just trying to get this to work but to accomplish what I want to I'd have to make more calls than the system allows me.
Is the way I am appending the lists at the end going to produce to output column/series I seek? The output I want looks like this:

# each list will be the length of the # of food items detected in the respective text data, the numbers
# being the calories of that food item.
[[23,23,22], [23,543,76, 322, 32, 1], [31,312,67,34,34,2], [31,543,76,32,12], [32,4,66,3], etc...]

Here is the code:

col = []
sem = []
while len(col) != len(data['post']):
    for i in data['post']:

        limit = 0
        calories = []
        for word in eval(i):
            if word not in food:
                continue
            else:
                print('Detected Word: ', word)
                querystring = {"ingr": "1 {}".format(word)}

                try: # ERROR HANDLING: CHECK FOR KEY ERROR
                    response = requests.request("GET", url, headers=headers, params=querystring)
                except KeyError as f:
                    print(f, 'Out of calls, restarting in 30 secs...')
                    time.sleep(30)
                    x += 1                      # move on to next API key as result of error
                    if ctrl == (len(keys) - 1):
                        x -= 2                  # move back to keys[0]: first API key
                        print(x)
                        print('Key', x, ':', key)
                else:
                    stats = response.text
                    print('Food Stats: \n', stats)

                    if 'status' or 'message' in eval(stats).keys(): # ERROR HANDLING: CHECKS FOR THESE KEY VALUES
                                                                    # TO DETECT RUNNING OUT OF API CALLS
                        print('Out of calls, restarting in 30 secs...')
                        time.sleep(30)
                        x += 1                               # move on to next API key as result of error
                        if ctrl == (len(keys) - 1):
                            x -= 2                           # move back to keys[0]: first API key
                        else:
                            print(x)
                            print('Key', x, ':', key)

                    else:
                        calories.append(eval(stats)['calories'])
                        limit += 1
                        print('Limit at {}'.format(limit))
                        if limit == 10:
                            x += 1
                            print(x)
                            if ctrl == (len(keys) - 1):
                                x -= 2
                                print('Reset x:', x)
                                print('Key', x, ':', key)
                        else:
                            print('Limit at {}, Halting program for 30 secs...'.format(limit))
                            limit = 0
                            time.sleep(30)
                            print('Resuming...')
                    sem.append(calories)
                col.append(sem)

Here are the variables involved with this code:

# dataset where we are getting text data from

data = pd.read_csv('mfp_1_proc_data.csv')


# Manual Food Bank to check if words are food.

food = ['waffle', 'apple', 'cottage', 'cheese', 'muffin', 'soup', 'coffee', 'butter', 'ham', 'syrup', 'tortilla',
        'salt', 'pepper', 'butternut', 'walnut', 'cinnamon', 'kale', 'spinach', 'banana', 'bean', 'turkey', 'jam',
        'carrot', 'cracker', 'rice', 'soy', 'tempura', 'sushi', 'orange', 'pasta', 'chicken', 'beef', 'fish', 'lamb',
        'pork', 'broccoli', 'lettuce', 'tomato', 'garlic', 'bacon', 'salad', 'vanilla', 'raspberry', 'cake', 'beer',
        'green', 'onion', 'vegetable', 'chocolate', 'sugar', 'chip', 'cookie', 'milk', 'cashew', 'vinegar', 'artichoke',
        'cheddar', 'cauliflower', 'pinto', 'egg', 'peanut', 'cocoa', 'avocado', 'pomegranate', 'clementine', 'grape']

# API keys to be cycled through operation

keys = ["16d7ecf46dmshc432a0e5b44478bp18c531jsn775dccbe9451",
        "e5d3101980msh4437c99f8467336p1f07a7jsne6206fb9828d",
        "f3d0adb9bamsh765dd46a0f7169ep1bbb31jsn68420afd160b"]

# variable for keys[x] to change which key is used

x = 0

# Dictionary to run as headers parameter for requests.request()
# 'x-rapidapi-key' value is meant to be changed to the next key in the keys list every 10 queries

headers = {
                'x-rapidapi-host': "edamam-edamam-nutrition-analysis.p.rapidapi.com",
                'x-rapidapi-key': keys[x]
            }

# useful variable to refer to headers key

key = headers['x-rapidapi-key']

# this should equal the index of whatever API key from the keys list key equals (if headers['x-rapidapi-key'] = keys[2],
# ctrl = 2)

ctrl = keys.index(key)

If this isn't the right forum to ask this type of question please let me know. If you guys can help with this, it'd be greatly appreciated because this data would help get my research project on a clear path to finish.

Let's start with this: "thought adding the else would ensure that all the code gets executed." - if you always want the code to be executed, don't put it in the else statement. Only use else if you only want the code to happen when the if conditions is false. What's your intent? — Elenchus, Jul 25 '20 at 19:14
The exception to that is when you don't want it to execute when the if condition is true, but there's no way to reach the code after the if statement - so you don't need the else after 'continue' because continue will start the next loop, and no other code in the current loop will run — Elenchus, Jul 25 '20 at 19:19
Also for the key not changing, although you're changing x you're not updating the header - x is a place holder for the value of 0 at the time the header is created, not a continual reference to a variable number, so you need to recreate the header with the new key each time x is changed — Elenchus, Jul 25 '20 at 19:28
@Elenchus Thanks for the responses. I thought I was doing something wrong with those else statements. For updating the key would that be as simple as `headers['x-rapidapi-key'] = keys[?]`? I put ? because I want the dict value to be the iterations of the keys list (So every 10 calls, the headers key value changes to the next element in the keys list). — Marcelino Velasquez, Jul 26 '20 at 19:51
Yep - ```headers['x-rapidapi-key'] = keys[x]``` but put it in the except block after you update the value of x. That way you're passing the new value of x in each time — Elenchus, Jul 27 '20 at 02:21

Refining the Code: Python Script for API and Creating a new column from it

0 Answers0