How to use urllib3.Retry?

Question

I have found next example

def prepare_retry_requester(retries: int = 5, forcelist: List = (503,)) -> requests.Session:
    requester = requests.Session()
    retry = urllib3.Retry(total=retries, backoff_factor=1, status_forcelist=forcelist)
    for protocol in 'http://', 'https://':
        requester.mount(protocol, requests.adapters.HTTPAdapter(max_retries=retry))
    return requester


with prepare_retry_requester(forcelist=[502, 503, 413]) as requester:
    response = requester.post(url, data=serialized)

But it still fails if i get 502 errors for a while (server is restarting for 10 seconds).

You are passing `max_retries` to HTTPAdapter which does not recover from 502 errors (by design) since it's not a connection error (but an applicative one). You can instead pass `retries` to `http.request`. See the [docs](https://urllib3.readthedocs.io/en/latest/user-guide.html) — Benjamin Gruenbaum, Dec 16 '19 at 08:27
If you need a custom timeout algorith, then subclass `urllib3.util.retry.Retry` and override method `get_backoff_time()`. For example: You can return a fixed number of fraction seconds, e.g., `5.75` — kevinarpe, Dec 14 '22 at 10:34

score 4 · Answer 1 · edited Dec 17 '19 at 12:12

4

You are passing max_retries to HTTPAdapter which does not recover from 502 errors (by design) since it's not a connection error (but an applicative one). You can instead pass retries to http.request. See the retrying docs.

edited Dec 17 '19 at 12:12

Quentin Pradet

4,691
2
29
41

answered Dec 17 '19 at 11:45

Benjamin Gruenbaum

270,886
87
504
504

How to use urllib3.Retry?

1 Answers1