Why are my API response always invalid ? Is there something wrong with my call_api function?[Solved]

Question

In my call_api function, I used a different way to detect invalid/valid APIs which was by checking if the string, "Hazard was successfully processed and at least one value is returned." was present in the header "Response_Description". Somehow my output is always false which is labelled N in the output table under the Pass column.

def call_api(url):
try:
    headers = {
        "User-Agent": "Mozilla/5.0 (X11; CrOS x86_64 12871.102.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.141 Safari/537.36"
    }
    response = requests.get(url, headers=headers)
    json_response = response.json()
    if (
        "Response_Description" in json_response
        and "Hazard was successfully processed" in json_response["Response_Description"]
    ):
        return True
    else:
        return False
except requests.exceptions.RequestException:
    return False

df = pd.read_csv(r"C:\Users\Jose.Moquaimbo\Bulk Calling APIs\dataset.csv")

# Number of iterations
num_iterations = 5

# Create an empty DataFrame to store the results
results_df = pd.DataFrame(columns=["Iteration", "Pass", "Time Taken"])

# Variables for tracking min, max, and total time
min_time = float("inf")
max_time = float("-inf")
total_time = 0


def process_iteration(iteration):
# Get a random sample of URLs from the DataFrame
 random_urls = df["url"].sample(n=1).tolist()

# Execute API calls concurrently using ThreadPoolExecutor
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    start_time = time.time()
    futures = [executor.submit(call_api, url) for url in random_urls]

# Wait for all futures to complete and get their results
results = [future.result() for future in futures]

# Stop timer
end_time = time.time()

# Calculate the time taken for this iteration
iteration_time = end_time - start_time

# Update min, max, and total time
global min_time, max_time, total_time
min_time = min(min_time, iteration_time)
max_time = max(max_time, iteration_time)
total_time += iteration_time

# Check if any API call was not successful in this iteration
passed = "Y" if all(results) else "N"

# Add the iteration results to the DataFrame
results_df.loc[iteration] = [iteration, passed, iteration_time]


# Run the iterations
for i in range(1, num_iterations + 1):
    process_iteration(i)

# Calculate average time per iteration
avg_time = total_time / num_iterations

# Display the results DataFrame
print(results_df)

# Summary statistics
print("Minimum time taken:", min_time)
print("Maximum time taken:", max_time)
print("Average time per iteration:", avg_time)
print("Y stands for error-free response and N for invalid response")

Output

       Iteration Pass  Time Taken
 1          1    N    0.276398
 2          2    N    0.298180
 3          3    N    0.307337
 4          4    N    0.323730
 5          5    N    0.333215
 Minimum time taken: 0.2763981819152832
 Maximum time taken: 0.33321452140808105
 Average time per iteration: 0.3077719688415527
 Y stands for error-free response and N for invalid response

Is there a way to modify the code, so it can correctly validate the API because it seems that the API is always invalidated. I suspect something is wrong my call_api function

Edit:

response_text = response.text

Instead of just returning False in case of an exception (which may be occurring) try reporting the actual exception detail. Also, make sure you always check the HTTP status code — DarkKnight, Jun 15 '23 at 06:59
Well I got rid of the exception detail and just left it as a True and False for if else statement but why is it not working still ? — Jose Moquiambo, Jun 15 '23 at 07:02
Okay I might have noticed my mistake. So the reason this wouldn't work and everything is returned false could be because response.json() is actually a dictionary and hence you can't just call out headers like that. Therefore im actually returning nothing, hence false — Jose Moquiambo, Jun 15 '23 at 07:12
Of course response.json() is a Python dictionary. What else were you hoping to get? — DarkKnight, Jun 15 '23 at 07:16
Okay i solved it. My solution is basically to turn the response.json() into a text form and just use an if-else statement to see if the text contains the desired string — Jose Moquiambo, Jun 15 '23 at 07:32

score 1 · Answer 1 · answered Jun 15 '23 at 07:13

Restructure your call_api() function to better handle exceptions and report any problems that might arise.

import requests
import sys

HEADERS = {
    'User-Agent': 'Mozilla/5.0 (X11; CrOS x86_64 12871.102.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.141 Safari/537.36'
}
KEY = 'Response_Description'
VALUE = 'Hazard was successfully processed'

def call_api(url: str) -> bool:
    try:
        with requests.get(url, headers=HEADERS) as response:
            response.raise_for_status()
            return VALUE in response.json().get(KEY, '')
    except Exception as e:
        print(e, file=sys.stderr)
    return False

Unfortunately, this doesn't seem to work because I'm still getting False for all my APIs. Could it be something wrong with this line, passed = "Y" if all(results) else "N". — Jose Moquiambo, Jun 15 '23 at 07:26

Why are my API response always invalid ? Is there something wrong with my call_api function?[Solved]

Output

1 Answers1