1

I'm using Python to to access a REST API that sometimes takes a long time to run (more than 5 minutes). I'm using pyelasticsearch to make the request, and tried setting the timeout to 10 minutes like this:

es = ElasticSearch(config["es_server_url"], timeout=600)
results = es.send_request("POST", 
                      [config["es_index"], "_search_with_clusters" ],
                      cluster_query)

but it times out after 5 minutes (not 10) with requests.exceptions.ConnectionError (Caused by <class 'socket.error'>: [Errno 104] Connection reset by peer)

I tried setting the socket timeout and using requests directly like this:

socket.setdefaulttimeout(600)
try:
    r = requests.post(url, data=post, timeout=600)
except:
    print "timed out"

and it times out after approximately 5 minutes every time.

How can I make my script wait longer until the request returns?

kielni
  • 4,779
  • 24
  • 21
  • 2
    That error means that the server closed the socket on you, so you specifying a longer timeout on your end isn't going to help. (It could also mean that the corporate web proxy you're really talking to has a timeout, or that some router in the middle has decided to cut you off. But what it can't be is your code timing out.) Is the server your code, or someone else's? – abarnert Oct 25 '13 at 20:33
  • Yes, it is my server. Thanks for the tip. I tried making the request directly to the actual server name:port and it succeeded after 8.5 minutes. It was the BigIP closing the connection after 5 minutes. – kielni Oct 25 '13 at 20:58

1 Answers1

1

The err "Connection reset by peer", aka ECONNRESET, means that the server—or some router or proxy between you and the server—closed the connection forcibly.

So, specifying a longer timeout on your end isn't going to make any difference. You need to figure out who's closing the connection and configure it to wait longer.

Plausible places to look are the server application itself, whatever server program drives that application (e.g., if you're using Apache with mod_wsgi, Apache), a load-balancing router or front-end server or reverse proxy in front of that server, or a web proxy in front of your client.

Once you figure out where the problem is, if it's something you can't fix yourself, you may be able to fix it by trickling from the server to the client—have it send something useless but harmless (an HTTP 100, an extra header, some body text that your client knows how to skip over, whatever) every 120 seconds. This may or may not work, depending on what component is hanging up.

abarnert
  • 354,177
  • 51
  • 601
  • 671