Flask blocking on endpoint without allowing another endpoint calls

Question

One of the endpoints of my flask API makes a long request to a Live Stream. Here is an example of the code:

@app.route('/stream')
def live_stream(sensor_id):
    stream = requests.get('stream_url', stream=True)
    return Response(stream_with_context(stream.iter_content(chunk_size=2048)),
                content_type=stream.headers['content-type'])

This route is working fine and the stream goes well. However, when I try to make requests to another routes it seems like the server get stucked at this endpoint.

I'm using a gevent WSGI server:

 http_server = WSGIServer(('0.0.0.0', 5000), app).serve_forever()

And I'm making the requests from a template already returned by a Flask route.

How can I make parallel requests to the API without get stucked on that?

Can you show/describe what's happening inside `stream_with_context`? — yorodm, Feb 12 '20 at 13:55
stream_with_context is a default method of Flask. The live stream is working file, but I can't make more requests. — Miguel Andrade, Feb 12 '20 at 14:08
Did you monkey-patch, as mentioned here: https://stackoverflow.com/questions/14551823/running-flask-gevent-requests-not-serving-concurrently — Maurice Meyer, Feb 16 '20 at 21:39

codeape · Accepted Answer · 2020-02-21T10:51:36.753

0

I have never used gevent, but if I understand correctly, it has a single-threaded event loop (like asyncio). But my understanding might be wrong.

Have a look at this answer https://stackoverflow.com/a/19153826/

When you have a piece of python code that takes a long time to run (for seconds) and does not cause switching of greenlets, all other greenlets / gevent jobs will 'starve' and have no computation time and it will look like your application 'hangs'.

I suggest to do one of the following:

Either, run your web app in a threaded WSGI server
Or, research if gevent has a network client that you can use instead of requests that is based on greelets
Or, run the blocking part of your code in a thread tool
Or, invoke from gevent import monkey; monkey.patch_all() as the very first thing in you server code, and see if that makes requests behave non-blocking.

edited Feb 21 '20 at 10:51

answered Feb 14 '20 at 08:21

codeape

97,830
24
159
188

But the requests module isn't from Gevent. It's this one https://pypi.org/project/requests/ . Do you have any suggest to use as alternative to Gevent? It must runs with Windows OS. – Miguel Andrade Feb 14 '20 at 13:40
Try first using flask’s development web server. You can pass an argument to `app.run()` to make it run multi-threaded. – codeape Feb 14 '20 at 16:48
1

It works it flask development server! Do you have any idea how can I set eventlet WSGI server to multithreading ? – Miguel Andrade Feb 14 '20 at 17:01
The eventlet WSGI server is a single threaded server, I believe. Your options are: Use a threaded server, or do the network request using a greenlets-based method, or do the request in a thread pool. – codeape Feb 19 '20 at 07:12
... added fourth option: check if gevent monkey-patching makes requests behave non-blocking. – codeape Feb 21 '20 at 10:56
@Miguel Andrade Since my answer a) correctly identifies the root cause and b) suggests several possible solutions, I suggest you mark it as accepted. – codeape Mar 15 '20 at 08:00

score 0 · Answer 2 · answered Mar 08 '20 at 08:27

You can check with the following steps:

make sure your code is monkey patched at the very first line(otherwise if some module is not patched and doing some I/O operation your app will block. I remember there is an API in gevent can hep to check monkey.is_module_patched('request'));
make sure you are not doing some CPU bounded task(CPU bounded code will block your code);

Hope it will help!

Flask blocking on endpoint without allowing another endpoint calls

2 Answers2