11

Up to now I followed this advice to reload the code:

https://code.google.com/archive/p/modwsgi/wikis/ReloadingSourceCode.wiki

This has the drawback, that the code changes get detected only every N second. I could use N=0.1, but this results in useless disk IO.

AFAIK the inotify callback of the linux kernel is available via python.

Is there a faster way to detect code changes and restart the wsgi handler?

We use daemon mode on linux.

Why code reload for mod_wsgi at all

There is interest in why I want this at all. Here is my setup:

Most people use "manage.py runserver" for development and some other wsgi deployment for for production.

In my context we have automated the creation of new systems and prod and development systems are mostly identical.

One operating system (linux) can host N systems (virtual environments).

Developers can use runserver or mod_wsgi. Using runserver has the benefit that it's easy for debugging, mod_wsgi has the benefit that you don't need to start the server first.

mod_wsgi has the benefit, that you know the URL: https://dev-server/system-name/myurl/

With runserver you don't know the port. Use case: You want to link from an internal wiki to a dev-system ....

A dirty hack to get code reload for mod_wsgi, which we used in the past: maximum-requests=1 but this is slow.

e4c5
  • 52,766
  • 11
  • 101
  • 134
guettli
  • 25,042
  • 81
  • 346
  • 663
  • Use ``inotify`` as you note and when notified send a ``SIGINT`` signal to yourself by getting process ID using ``os.getpid()``. – Graham Dumpleton Jun 20 '16 at 07:32
  • what's wrong with just doing `touch wsgi.py` after updating the code? – e4c5 Jun 21 '16 at 09:52
  • @e4c5 Why I don't like to touch wsgi.py by hand: I am lazy and I want to do as less as possible, since this makes me faster. – guettli Jun 21 '16 at 10:25
  • it might make you faster but it might make the reload process a lot slower! Have you considered using something like fabric to automate your site updates? That's definitely a lazy way of doing it – e4c5 Jun 21 '16 at 10:46
  • @e4c5 I don't want this for production. I want it for testing. BTW, for production we are switching from fabric to salt. Why should the reload process get slower? Inotify does nothing, expect a file changes. That's why I want to switch from checking the mtime for ~2k files every second to a different solution. – guettli Jun 21 '16 at 11:10
  • Well, I thought this was production and was thinking about the situation when you update 213 files with a git pull or rsync. That's going to fire a lot of events. – e4c5 Jun 21 '16 at 11:48
  • If it is local, why are you not using manage.py runserver ? – e4c5 Jun 21 '16 at 12:15
  • @e4c5 most automated tests use the Client class provided by django or selenium. But some tests (more integration than unittests) check the wsgi-setup. – guettli Jun 21 '16 at 12:22
  • what is the editor/IDE you are using? It's quite possible that this maybe doable in that – e4c5 Jun 21 '16 at 23:38
  • Also why does the wsgi setup need to be tested from inside django? that's most unusual. Can you please update the question with this and other information. The fact that you haven't got an answer yet I think is clear indication that your question or it's purpose hasn't been fully understood by viewers – e4c5 Jun 22 '16 at 06:39
  • @e4c5 I updated the question and explained my background. Please ask if you don't get it. – guettli Jun 22 '16 at 07:57
  • Regarding "With runserver you don't know the port", I'm pretty sure you can specify the port in a parameter. – Aya Jun 25 '16 at 14:23

2 Answers2

4

Preliminaries.

Developers can use runserver or mod_wsgi. Using runserver has the benefit that you it easy for debugging, mod_wsgi has the benefit that you don't need to start the server first.

But you do, the server needs to be setup first and that takes a lot of effort. And the server needs to be started here as well though you can configure it to start automatically at boot.

If you are running on port 80 or 443 which is usually the case, the server can be started only by the root. If it needs to be restarted you will have to ask the super user's help again. So ./manage.py runserver scores heavily here.

mod_wsgi has the benefit, that you know the URL: https://dev-server/system-name/myurl/

Which is no different from the dev server. By default it starts on port 8000 so you can access it as http://dev-server:8000/system-name/myurl/. If you wanted to use SSL with the development server you can use a package such as django-sslserver or you can put nginx in front of django development server.

With runserver you don't know the port. Use case: You want to link from >an internal wiki to a dev-system ....

With runserver, the port is well defined as mentioned above. And you can make it listen on a different port for exapmle with:

 ./manage.py runserver 0.0.0.0:9090

Note that if you put development server behind apache (as a reverse proxy) or NGINX, restarting problems etc that I have mentioned above do not apply here.

So in short, for development work, what ever you do with mod_wsgi can be done with the django development server (aka ./manage.py runserver).

Inotify

Here we are getting to the main topic at last. Assuming you have installed inotify-tools you could type this into your shell. You don't need to write a script.

  while inotifywait -r -e modify .; do sudo kill -2 yourpid ; done

This will result in the code being reloaded when ...

... using daemon mode with a single process you can send a SIGINT signal to the daemon process using the ‘kill’ command, or have the application send the signal to itself when a specific URL is triggered. ref: http://modwsgi.readthedocs.io/en/develop/user-guides/frequently-asked-questions.html#application-reloading

alternatively

while inotifywait -r -e modify .; do touch wsgi.py ; done

when

... using daemon mode, with any number of processes, and the process reload mechanism of mod_wsgi 2.0 has been enabled, then all you need to do is touch the WSGI script file, thereby updating its modification time, and the daemon processes will automatically shutdown and restart the next time they receive a request.

In both situations we are using the -r flag to tell inotify to monitor subdirectories. That means each time you save a .css or .js file apache will reload. But without the -r flag changes to python code in subfolders will be undetected. To have the best of both worls, remove css, js, images etc with the --exclude directive.

What about when your IDE saves an auto backup file? or vim saves the .swp file? That too will cause a code reload. So you would have to exclude those file types too.

So in short, it's a lot of hard work to reproduce what the django development server does free of charge.

e4c5
  • 52,766
  • 11
  • 101
  • 134
3

You can use inotify hooktables to run any command you want depending on a i-notify signal (here's my source link: http://terokarvinen.com/2016/when-files-change-take-action-inotify-hookable).

After looking the tables you can just reload the code of apache.

For your specific problem, it should be something like: 

inotify-hookable --watch-directories sources/ --recursive --on-modify-command './code_reload.sh'

In the previous link, the command to execute was just a simple touch flask/init.wsgi

So, the whole code (adding ignored files was): 

inotify-hookable --watch-directories flask/ --recursive  --ignore-paths='flask/init.wsgi' --on-modify-command 'touch flask/init.wsgi'

As stated here: Flask + mod_wsgi automatic reload on source code change, if you have enabled WSGIScriptReloading, you can just touch that file. It will cause the entire code to reload (not just the config file). But, if you prefer, you can set any other script to reload the code.

After googling a bit, it seems to be a pretty standard solution for that problem and I think that you can use it for your application.

Community
  • 1
  • 1
Alexis Clarembeau
  • 2,776
  • 14
  • 27