Return status code when maximum retries attempted using backoff

Question

Using asyncio and aiohttp, I have implemented an async function that triggers an API get request whenever a new record is inserted into database. If the request is successful, the status code has to be updated in database, otherwise the request should be retried 4 times and if it still fails, then the status code has to be updated in database.

To raise the exception on 404 status code, I have added raise_for_status flag to the aiohttp client session. When exception arises, the backoff decorator will retry the API call 4 times and when it still fails, it doesn't return any status code. This is what I have done:

# backoff decorator to retry failed API calls by "max_tries"
@backoff.on_exception(backoff.expo, aiohttp.ClientResponseError, max_tries=4, logger=logger)
async def call_url(language: str, word:str, headers:dict) -> bytes:
      url = f"https://od-api.oxforddictionaries.com:443/api/v2/entries/{language}/{word.lower()}"
      print(f"Started: {url}")

      # Create aiohttp session to trigger 'get' dictionary API call with app_id, app_key as headers
      async with aiohttp.ClientSession(headers=headers) as session:   

          # raise_for_status is used to raise exception for status_codes other than 200      
          async with session.get(url, raise_for_status=True) as response:

              # Awaits response from dictionary API  
              content = await response.read()                              
              status = response.status
              print("Finished: ", status)

              # Returns API status code to be updated in db
              return status

I can't add the try-except clause because once exception is raised, it's handled by the try-except clause and the backoff doesn't retry the failed API call. Is there a way to make backoff decorator return the status code after maximum retries are attempted?

score 1 · Accepted Answer · answered Nov 20 '22 at 05:23

You say that "when it still fails it doesn't return any status code." But it has to either return something or else raise an exception. How about wrapping the call_url and handling the error situation in the wrapper function? The wrapper function always returns a status code. For instance:

   async def my_call_url(language: str, word:str, headers:dict) -> bytes:
      try:
          return await call_url(language, word, headers)
      except WhateverExceptionAiohttpRaises:
          return some_status_code

If, instead of raising, the decorator returns None, you can make the appropriate changes.

Your code will now call this new function instead of the other one.

Return status code when maximum retries attempted using backoff

1 Answers1