Timeout for xmlrpclib client requests

Question

I am using Python's xmlrpclib to make requests to an xml-rpc service.

Is there a way to set a client timeout, so my requests don't hang forever when the server is not available?

I know I can globally set a socket timeout with socket.setdefaulttimeout(), but that is not preferable.

score 12 · Accepted Answer · edited Apr 23 '15 at 07:24

12

The clean approach is to define and use a custom transport, e.g.: ! this will work only for python2.7 !

import xmlrpclib, httplib

class TimeoutTransport(xmlrpclib.Transport):
    timeout = 10.0
    def set_timeout(self, timeout):
        self.timeout = timeout
    def make_connection(self, host):
        h = httplib.HTTPConnection(host, timeout=self.timeout)
        return h

t = TimeoutTransport()
t.set_timeout(20.0)
server = xmlrpclib.Server('http://time.xmlrpc.com/RPC2', transport=t)

There's an example of defining and using a custom transport in the docs, though it's using it for a different purpose (access via a proxy, rather than setting timeouts), this code is basically inspired by that example.

edited Apr 23 '15 at 07:24

Cédric

817
10
11

answered Mar 11 '10 at 15:37

Alex Martelli

854,459
170
1,222
1,395

this example don't work on python 2.5, 2.6 (TimeoutTransport instance has no attribute 'proxy') also httplib.HTTP don't have timeout value. – Ib33X Nov 10 '10 at 14:41
1

I noticed that the original `Transport.make_connection` method in Python 2.7 has some more logic to handle keep-alive, extra headers, etc., and that it also uses `httplib.HTTPConnection` and not `httplib.HTTP`. I don't know is that important in typical use cases, but I decided to copy the original code and change it to use timeouts like shown in the above answer. – Pekka Klärck Sep 23 '14 at 20:45

user755236 · Answer 2 · 2011-05-16T08:52:20.680

doh, to make this work in python2.6+ do this:

class HTTP_with_timeout(httplib.HTTP):
    def __init__(self, host='', port=None, strict=None, timeout=5.0):
        if port == 0: port = None
        self._setup(self._connection_class(host, port, strict, timeout=timeout))

    def getresponse(self, *args, **kw):
        return self._conn.getresponse(*args, **kw)

class TimeoutTransport(xmlrpclib.Transport):
    timeout = 10.0
    def set_timeout(self, timeout):
        self.timeout = timeout
    def make_connection(self, host):
        h = HTTP_with_timeout(host, timeout=self.timeout)
        return h

score 5 · Answer 3 · answered Jun 28 '11 at 16:40

Why not:

class TimeoutTransport(xmlrpclib.Transport):

def setTimeout(self, timeout):
    self._timeout = timeout

def make_connection(self, host):
    return httplib.HTTPConnection(host, timeout=self._timeout)

?

After all, HTTP and HTTPS seem to be no more than compatibility classes for older Python versions.

score 3 · Answer 4 · answered Jun 06 '13 at 20:19

An alternative implementation that would be compatible with python 2.7 would be as follows (with a comment containing what you would want if you're using python 2.6):

import socket
import xmlrpclib

class TimeoutTransport (xmlrpclib.Transport):
    """
    Custom XML-RPC transport class for HTTP connections, allowing a timeout in
    the base connection.
    """

    def __init__(self, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, use_datetime=0):
        xmlrpclib.Transport.__init__(self, use_datetime)
        self._timeout = timeout

    def make_connection(self, host):
        # If using python 2.6, since that implementation normally returns the 
        # HTTP compatibility class, which doesn't have a timeout feature.
        #import httplib
        #host, extra_headers, x509 = self.get_host_info(host)
        #return httplib.HTTPConnection(host, timeout=self._timeout)

        conn = xmlrpclib.Transport.make_connection(self, host)
        conn.timeout = self._timeout
        return conn

# Example use
t = TimeoutTransport(timeout=10)
server = xmlrpclib.ServerProxy('http://time.xmlrpc.com/RPC2', transport=t)

Using the super-method would allow the underlying 2.7 implementation to maintain its HTTP/1.1 keep-alive functionality it defines.

A thing to note is that if you're trying to use XML-RPC over an https connection/address, replace xmlrpc.SafeTransport references with xmlrpc.Transport instead, and, if you're using the 2.6 implementation, use httplib.HTTPSConnection.

score 0 · Answer 5 · answered May 26 '20 at 13:31

If anyone is trying to do this in Python 3+ and makes use of the context kwarg (In my case to allow connection to self-signed SSL certs), the following code works for me

class TimeoutTransport (xmlrpc.client.SafeTransport):
    def __init__(self, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, context=None, use_datetime=0):
        xmlrpc.client.Transport.__init__(self, use_datetime)
        self._timeout = timeout
        self.context = context

    def make_connection(self, host):
        conn = xmlrpc.client.SafeTransport.make_connection(self, host)
        conn.timeout = self._timeout
        return conn

And then call with:

 url = "https://localhost:8080/RPC2"
 t = TimeoutTransport(timeout=2, context=ssl._create_unverified_context())
 xml_conn = xmlrpc.client.ServerProxy(
        url,
        transport=t
    )

Timeout for xmlrpclib client requests

5 Answers5