Do something on connection error apart from retry with python requests

Question

I am using Python requests to make a post request.
I am trying to do something like this as shown in below post: Retry with requests

When there is connection error or response status code received is from status_forcelist, it should retry(which is working fine). What I want to do is after first try (before retrying), I want to do some other stuff. This can be possible if I can catch Exception and handle it to do other stuff. But it seems that requests is not raising any exception in case of connection error or response code is in status_forcelist unless retry count reaches to max configured. How can I achieve this?

Here is the code sample:

def requests_retry_session(
    retries=3,
    backoff_factor=0.3,
    status_forcelist=(500, 502, 504),
    session=None,
):
    session = session or requests.Session()
    retry = Retry(
        total=retries,
        read=retries,
        connect=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
    )
    adapter = HTTPAdapter(max_retries=retry)
    session.mount('http://', adapter)
    session.mount('https://', adapter)
    return session

def do_something_more():
    ## do something to tell user API failed and it will retry
    print("I am doing something more...")

Usage...

t0 = time.time()
try:
    response = requests_retry_session().get(
        'http://localhost:9999',
    )
except Exception as x:
    # Catch exception when connection error or 500 on first attempt and do something more

    do_somthing_more() 
    print('It failed :(', x.__class__.__name__)
else:
    print('It eventually worked', response.status_code)
finally:
    t1 = time.time()
    print('Took', t1 - t0, 'seconds')

I know exception will be raised after max allowed attempts(defined in retries=3). All I want is some signal from requests or urllib3 to tell my main program that first attempt is failed and now it will start retrying. So that my program can do something more based on it. If not through exception, something else.

You should try to come up with a few lines of code alone to show you are actively thinking about solving your problem and then reach out to other people asking for help. Even with providing the site your problem can be unclear unless you show what you really want by pasting what you got so far here. — Sqoshu, Jan 02 '18 at 14:15
@Sqoshu That make sense. I have updated my question with code sample as you suggested to make it more clear. — rsnhah, Jan 02 '18 at 14:53

Sqoshu · Answer 1 · 2018-01-03T21:09:26.387

The most robust way (but probably not the best and certainly not the most efficient) would be just to set the retries to 0 - then the exception will be raised every time. Then I would just call the function three times, with manual counter, which will count how many times you tried to reconnect. Something like this (I didn't check if it works, just wanted to show you my way of thinking):

counter = 0
t0 = time.time()
for i in range(3):
    try:
        response = requests_retry_session().get(
            'http://localhost:9999',
        )
        #This should already be set to retries=0
    except MaxRetryError:
            counter += 1
            do_something_more() 
            print('It is the {} time it failed'.format(counter))
    else:
         break #If there isn't MaxRetryError, it connected successfully, so we don't have to execute for anymore
t1 = time.time()
print('Took', t1 - t0, 'seconds')

Do something on connection error apart from retry with python requests

1 Answers1