How do I get retry handling with python zeep? I'm using a requests retry session, but the exception is not handled

Question

I'm using zeep. The service I am using generates timeout errors every now and then, and I want to use automatic retry functionality.

I am trying to use a Requests retry session, but the timeout exception is not being caught and no retry is attempted.

I set up a Requests retry session (below) and I set up the client attribute of my class (currently with a timeout value to cause errors):

 session = requests_retry_session()
 transport = Transport(session=session,timeout=20,operation_timeout=.001)
 self.client = Client(self.wsdl_url,transport=transport)

...

def requests_retry_session(
        retries=10,
        backoff_factor=0.3,
        status_forcelist=(500, 502, 503, 504),
        session=None,
) -> requests.Session:
    session = session or requests.Session()
    retry = Retry(
        total=retries,
        read=retries,
        connect=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
    )
    adapter = HTTPAdapter(max_retries=retry)
    session.mount('http://', adapter)
    session.mount('https://', adapter)
    return session

I'm getting a requests.exceptions.ReadTimeout (I assume this error is the same as the one I'm getting in real life, but with a 20s timeout, my logging is not good enough yet to be sure)

The exception is not being handled by the session.

Why not?

EDIT: zeep sends a POST request somewhere, which I did not find with my own debugging but activating debug logging in Requests shows it.

Retry does not work on POST by default.

https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html

I set method_whitelist=False to change that, and now I am triggering retries.

Ah. requests debugging shows that there are POSTs in use by zeep, so I will try method method_whitelist=False when making the retry sessions, since POSTs are not retried by default . — Tim Richardson, Sep 20 '19 at 03:42
Hey there! Sorry for the late poke, but have you found any solution to your problem? I was browsing old questions, saw your comment, and wondered if you had answered your own question - if so, please consider posting it as an answer and accepting it. Thanks, and good luck! — Nick Reed, Nov 01 '19 at 15:31
Yes, set method_whitelist=False which will trigger retries when POSTs fail. Be careful; you need to know if repeated POSTs to your endpoint are a bad idea. Enable it selectively. — Tim Richardson, Nov 04 '19 at 05:05

Tim Richardson · Accepted Answer · 2019-11-25T22:31:34.163

set method_whitelist=False which will trigger retries when POSTs fail. Be careful; you need to know if repeated POSTs to your endpoint are a bad idea.

Here is some code setting up a retry session with requests:

from zeep import Client
from zeep.transports import Transport
import requests
from requests.adapters import HTTPAdapter, Retry

def requests_retry_session(
        retries=5,
        backoff_factor=0.3,
        status_forcelist=(500, 502, 503, 504),
        session=None,**kwargs) -> requests.Session:
    session = session or requests.Session()
    retry = Retry(
        total=retries,
        read=retries,
        connect=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
        **kwargs
        )
    adapter = HTTPAdapter(max_retries=retry)
    session.mount('http://', adapter)
    session.mount('https://', adapter)
    return session

And then to make an instance of the Zeep client which will retry on POST fails

 session = requests_retry_session(method_whitelist=False)
 transport = Transport(session=session,timeout=30,operation_timeout=30)
 zeep_client = Client(your_wsdl_url,transport=transport)

How do I get retry handling with python zeep? I'm using a requests retry session, but the exception is not handled

1 Answers1