1

I'm running multiple Django sites on the same Apache instance under mod_wsgi. Currently my apache.conf files contain the following directives (no WSGIApplicationGroup specified):

WSGIDaemonProcess mysite   \
    display-name=mysite    \
    threads=50             \
    maximum-requests=10000 \
    umask=0002             \
    home=/srv/www/mysite   \
    python-path=/srv/www:/srv/src:/srv/venv/prod/lib/python2.7/site-packages \
    python-eggs=/srv/.python-eggs

WSGIProcessGroup mysite
WSGIScriptAlias / /srv/www/mysite/wsgi.py

I touch /srv/www/mysite/wsgi.py whenever I need to reload the site, and it causes a noticeable freeze in all clients.

After reading https://groups.google.com/forum/#!topic/modwsgi/QJkt5UWYpss it sounds like I can get rid of the "reload pause", by specifying process/application groups in the WSGIScriptAlias directive:

WSGIDaemonProcess mysite   \
    display-name=mysite    \
    threads=50             \
    maximum-requests=10000 \
    umask=0002             \
    home=/srv/www/mysite   \
    python-path=/srv/www:/srv/src:/srv/venv/prod/lib/python2.7/site-packages \
    python-eggs=/srv/.python-eggs

WSGIScriptAlias / /srv/www/mysite/wsgi.py    \
    process-group=mysite                     \
    application-group=mysite

IIUC, I need to provide both process-group= and application-group= for the preloading to happen.

All the docs I've found so far uses application-group=%{GLOBAL}, but that seems wrong for my use case, where each virtual host should run code based on the individual site's settings.py file (correct?).

Should I use the predefined %{RESOURCE} variable instead of mysite.

Can I share the same application-group between the http and https versions of the same site? (I know I can't do that with the process group).

thebjorn
  • 26,297
  • 11
  • 96
  • 138

1 Answers1

3

Each virtual host Django site should use a separate daemon process group, so application-group of %{GLOBAL} is fine as it is forcing the use of the main interpreter context within the respective process groups. It is not shared across process groups.

Do note that preloading isn't going to necessarily help too much if you are doing restarts when the site is under heavy load as things will still need to wait for the process to start and load the application.

Having threads=50 looks to be quite excessive. What throughout do you get and what is your average response time. Best performance is achieved by using 3-5 threads per process and using multiple processes. Using multiple processes obviously means using more memory though as there will be multiple copies of your application.

Finally, yes it is recommended, unless you have a good reason otherwise, to have both HTTP and HTTPS versions of site delegated to the same daemon process group. Specify the WSGIDaemonProcess in the first VirtualHost as seen for that ServerName by Apache. In the second in the 80/443 pair, don't have a WSGIDaemonProcess and refer to the named process group in the other VirtualHost context. This reaching across is allowed where ServerName is the same.

Graham Dumpleton
  • 57,726
  • 6
  • 119
  • 134
  • Thanks for the info. Our response times are usually in the 300-600ms range for fully generated pages. We can handle sustained loads of 30 req/sec with bursts of twice that as long as they don't last more than a couple of seconds. The reason we went single process was due to memory issues when we tried to run @ 4core/5GB. I doubled that last week, so perhaps it's time to look at those parameters again.. – thebjorn Apr 28 '15 at 18:39
  • What is the actual size of your memory for the web application process? Ensure that you are using Apache 2.4 and recent version of mod_wsgi. Older Apache 2.2 has memory usage issues in certain cases which Apache 2.4 addresses, but also with latest mod_wsgi versions also have changes to avoid issues in Apache 2.2 and 2.4 which can cause extra memory usage. In short, do not use out of date versions of Apache and mod_wsgi shipped with Linux distros. – Graham Dumpleton Apr 28 '15 at 21:59
  • The web processes (n=17) have avg rss=97000 (max 239000) and avg vsz=833000 (max 1 113 000). We're using default Ubuntu 12.04, but I'm looking into upgrading. Our ISP filed a question on launchpad (https://answers.launchpad.net/ubuntu/+source/mod-wsgi/+question/266078), although that seems to be focused on the security aspect rather than performance... – thebjorn Apr 29 '15 at 07:42
  • Where does n=17 come from? Do you mean you have 17 separate Django instances, each in their own mod_wsgi daemon process group? Your configuration only shows default of single mod_wsgi daemon process. Or are you looking at both Apache child worker processes as well as mod_wsgi daemon processes? FWIW, the mod_wsgi with 12.04 will exhibit excess memory usage if you are dealing with large file uploads and slow HTTP clients. Why it is important to use latest versions. Linux distros don't back port such fixes. – Graham Dumpleton Apr 29 '15 at 11:06
  • At least one of our virtualhosts is dealing with photo uploads from mobiles, so I'm guessing we're hitting the issue. We're running 17 different sites/virtualhosts that use mod_wsgi (and a handful that don't) on this server. I counted/tabulated the output of `ps gaux` for the rows matching the `display-name` that is set in the `WSGIDaemonProcess` directive. We'll be upgrading mod_wsgi after the weekend (May 1 being labor day, and a public holiday, everyone is off). – thebjorn Apr 29 '15 at 11:28