1

I created an python api on openshift online with python image. If you request all the data, it takes more than 30 seconds to respond. The server gives a 504 gateway timeout http response. How do you configure the length a response can take? > I created an annotation on the route, this seems to set proxy timeout.

haproxy.router.openshift.io/timeout: 600s

Problem remains, I now got logging. It looks like the message comes from mod_wsgi.

I want to try alter the configuration of the httpd (mod_wsgi-express process) from request-timeout 60 to request-timeout 600. Where doe you configure this. I am using base image https://github.com/sclorg/s2i-python-container/tree/master/2.7

Logging: 

Timeout when reading response headers from daemon process  'localhost:8080':/tmp/mod_wsgi-localhost:8080:1000430000/htdocs

Does someone know how to fix this error on openshift online

jeew
  • 21
  • 4
  • What sort of work are you doing that requests take that long? It sounds like you should not be doing the processing in the context of a web request, but offloading the processing to a task queueing system such as Celery. By allow such long running requests, if you get many of these at the same time, you will block up the whole web server as you will run out of capacity. – Graham Dumpleton Nov 15 '17 at 19:39
  • I agree, that could be a problem in the future. For now it works because the request is made max 10 times a day, but this can increase in the future. It is an api request with a lot of work serverside to generate the data. What would your solution be? Offload the work to celery and send the response when finished? – jeew Nov 16 '17 at 08:00
  • Celery would be overkill if so infrequent. But usually you would offload to background and return acknowledgement that queued. Then have client poll somehow to check if complete. If that works for you currently go with it. The only concern would have is that those time outs then apply to all requests handled, which means for the short requests you loose ability to mod_wsgi to recover if they start getting stuck. One can start using multiple daemon process groups and split traffic as explained in http://blog.dscpl.com.au/2014/02/vertically-partitioning-python-web.html – Graham Dumpleton Nov 16 '17 at 08:08
  • Using multiple daemon processes can be something look at later if is an issue. Can then explain how to do that with mod_wsgi-express. – Graham Dumpleton Nov 16 '17 at 08:09
  • Great, a daemon process for this endpoint sounds like a solution, I would lik to know how this works, will read the link you send me. I am thinking also on another solution. The data changes one time per day in the database, I could also generate all the possible data, save it in a file. And let the api work with the file. What would you prefer. – jeew Nov 16 '17 at 08:13
  • Generating it once per day sounds even better. Doing that in the web process is still not the best idea, but one can supply a service script to mod_wsgi-express which can be used to start a separate managed process. That can then use a cron package for Python to schedule the action to generate it. I have shown someone how to do this recently. Will just need to remember where. – Graham Dumpleton Nov 16 '17 at 10:16
  • See ``--service-script`` option in https://github.com/openshift-katacoda/blog-django-py/blob/master/app.sh#L28 and then the service script using ``schedule`` package in https://github.com/openshift-katacoda/blog-django-py/blob/master/cronjobs.py – Graham Dumpleton Nov 16 '17 at 10:19
  • That ``powershift image jobs hourly`` command is just triggering scripts in a directory. You could just have your Python code execute what is needed directory in that ``cronjobs.py`` file. – Graham Dumpleton Nov 16 '17 at 10:21
  • @GrahamDumpleton you are the best, thank you. If I can help you in any way in the futute (don't know how) just let me know, cheers! – jeew Nov 16 '17 at 11:08
  • Only thing can suggest, is if need help in future and expect a discussion will ensue like this time, use the OpenShift group at https://groups.google.com/forum/#!forum/openshift Works better than SO. – Graham Dumpleton Nov 16 '17 at 11:11

1 Answers1

1

Next to alter timeout of haproxy of the route of my app

haproxy.router.openshift.io/timeout: 600s

I altered the request-timeout and socket-timeout in app.sh of my python application. So the mod_wsgi-express server is configured with a higher timeout

ARGS="$ARGS --request-timeout 600"
ARGS="$ARGS --socket-timeout 600"

My application now wait 10 minutes before cancelling a request

jeew
  • 21
  • 4