How to debug basic issues configuring django to be served with apache and mod-wsgi?

Question

It is common that people have trouble setting up django to be serves by apache and mod-wsgi. The common symptom is "Import error" ... for some reason (generally slightly different in each case) settings.py or similar doesn't import (see "Related" in the right column on this page for numerous examples!).

I have read through the other questions on this topic, and none appear to have a solution that works for my situation (one was a basic misconfig by the poster identified by an answerer - I don't appear to have this issue, others apply to using wsgi from other modules etc).

When apache/mod-wsgi fails to load, how can you debug it?

What can you do to make something give you a better message than "Import error"?

Obviously, I'm looking for a method that will identify what's wrong in my case. But I really would like to know how to approach debugging this sort of failure: there seems to be no way that I can find to get information about what it is causing it to fail.

In my case, I am trying to do what appears to be a straightforwards django deploy with mod-wsgi - really by the book, the same as the doc, but get the error:

ImportError: Could not import settings 'cm_central.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named cm_central.settings

I can't see why it can't find this module.

/home/cmc/src/cm_central/cm_central/settings.py exists, can be loaded by pythonn without error, and in fact works OK with ./manage.py runserver.

Is it possible that there is some import error occurring in the context of apache, that doesn't occur when I load it myself? I'm wondering because of the words "Is there an import error in the settings file?" ... why does it ask that? If there were an import error, how would I debug it?

I have this in /etc/apache2/sites-enabled/cm-central:

<VirtualHost *:80>

  WSGIScriptAlias / /home/cmc/src/cm_central/cm_central/wsgi.py
  WSGIDaemonProcess cm-central.johalla.de python-path=/home/cmc/src/cm_central:/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages
  WSGIProcessGroup cm-central.johalla.de

  <Directory /home/cmc/src/cm_central/cm_central>
    <Files wsgi.py>
      Order deny,allow
      Allow from all
    </Files>
  </Directory>

</VirtualHost>

And this in wsgi.py: (it I haven't modified it from what django generated)

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cm_central.settings")

from django.core.wsgi import get_wsgi_application
from dj_static import Cling

application = Cling(get_wsgi_application())

The full traceback is:

[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] mod_wsgi (pid=10273): Target WSGI script '/home/cmc/src/cm_central/cm_central/wsgi.py' cannot be loaded as Python module.
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] mod_wsgi (pid=10273): Exception occurred processing WSGI script '/home/cmc/src/cm_central/cm_central/wsgi.py'.
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] Traceback (most recent call last):
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/src/cm_central/cm_central/wsgi.py", line 16, in <module>
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     application = Cling(get_wsgi_application())
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     django.setup()
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/__init__.py", line 20, in setup
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/conf/__init__.py", line 46, in __getattr__
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     self._setup(name)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/conf/__init__.py", line 42, in _setup
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     self._wrapped = Settings(settings_module)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/conf/__init__.py", line 98, in __init__
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     % (self.SETTINGS_MODULE, e)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] ImportError: Could not import settings 'cm_central.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named cm_central.settings
[Sun Nov 09 12:04:06 2014] [notice] Graceful restart requested, doing restart
[Sun Nov 09 12:04:06 2014] [notice] Apache/2.2.22 (Debian) PHP/5.4.4-14+deb7u14 mod_wsgi/3.3 Python/2.7.3 configured -- resuming normal operations
[Sun Nov 09 21:34:15 2014] [error] Not Found: /

If an answer could help me spot what's wrong, that'd be great - but even better is an answer to "how to debug this". How does one find out why the settings.py won't load?

score 1 · Answer 1 · edited Nov 22 '14 at 05:07

1

there could be quite different reasons for your issue.

permissions - maybe your wsgi daemon process is running under a different user/group than when you interactively try with python manage.py runserver. maybe the permissions set on your files make a difference then. maybe try to interactively use same user as for the daemon or vice versa. mod_wsgi has nice options so you can run your daemons under separate users, which is good for security also due to separation.

sys.path could be different (although it somehow looks correct in your daemon config), the cwd could be different.

as you seem to use a virtualenv you maybe also want to follow the special setup instructions for Mod-wsgi and Virtualenvs

edited Nov 22 '14 at 05:07

BomberMan

1,094
3
13
33

answered Nov 16 '14 at 16:54

Thomas Waldmann

501
2
7

Yes - these are all potential reasons, and hence my question "how can I debug the situation?". One way is to look at things like these in turn, and guess whether they are right. I am kind of hoping for something more "deterministic" - some way of getting an error message from somewhere that tells me what's wrong... another thing I suspet that you haven't mentioned is the configuration of the database, and another one is DOCUMENT_ROOT. All these things could be wrong: how can I coax django or apache to issue a better message to debug it? – GreenAsJade Nov 17 '14 at 06:47
(Note: the django docs had some advice about virtualenv, but I need to check the page you linked to as well - thanks, will do) – GreenAsJade Nov 17 '14 at 06:54

rparent · Answer 2 · 2014-11-18T16:50:23.307

You could try adding this to your wsgi.py:

path = os.path.join(os.path.dirname(__file__), "..") # Adapt the path here to match the root of your django project
if path not in sys.path:
    sys.path.append(path)

to ensure that your Django project is in the python path ?

If an answer could help me spot what's wrong, that'd be great - but even better is an answer to "how to debug this". How does one find out why the settings.py won't load?

It's hard to answer that question if we don't know exactly what you have already tried. Based on the traceback (Is it on sys.path? Is there an import error in the settings file?), I'd say :

Check that your django project base folder is in the python path (which I haven't read you have done, that's why I made the suggestion above)
Check that there are no exeptions raised when loading your settings.py file (which you already did)

Thanks for these suggestions, I will try them. What I'm really looking for, though (what the question is asking for) us "how do I get a better error message other than 'could not import settings.py'?". How do I make something tell me _why_ it could not import it? If we could answer this question, reams of similar questions would be solved! — GreenAsJade, Nov 18 '14 at 21:31

andzep · Answer 3 · 2014-11-19T00:15:46.773

First, to the real question here. How to get more information about what exactly is failing? I might be wrong, but I'm almost sure there is no tools/packages/logs or whatever which would give you more infos on the problem than what you've already got: the traceback. So I guess the only way of debugging this errors, is the "traceback" method ;-)

--- Now to the problem:

Maybe it's a permission problem. In your WSGIDaemonProcess you don't specify any user or group, and maybe apache is not allowed to read/execute the files. Can you try adding user and group to this line like this:

    WSGIDaemonProcess cm-central.johalla.de user=<username> group=<username> python-path=/home/cmc/src/cm_central:/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages

Also, you're not using a DocumentRoot. Maybe that's why it's not finding the right path? In my vhosts, I always include a DocumentRoot, which has the same path as the Directory but with a leading slash, in your case:

<VirtualHost *:80>
    DocumentRoot /home/cmc/src/cm_central/cm_central/

So maybe that solves your problem.

score 1 · Answer 4 · answered Nov 19 '14 at 14:25

The error message is pretty explicit already -- mod_wsgi cannot find your settings file (line 2, os.environ.setdefault), check your system path (I understand that Django can find your settings file when you use runserver, but mod_wsgi cannot). To check your system path, you can modify the wsgi.py file and print / log the sys.path at the very top of the file:

import sys
print sys.path

In addition to the virtual env docs pointed to by @Thomas Waldmann, you may also need to manually append the project directory into the sys.path, as mentioned here in the Django config docs. Example code I use is below:

ALLDIRS = ['/usr/local/pythonenv/assessments/lib/python2.6/site-packages']

import os
import sys
import site

# from https://code.google.com/p/modwsgi/wiki/VirtualEnvironments

sys.path.insert(0, '/var/www/assessments/assessments-main/')  # settings.py file here
sys.path.insert(1, '/var/www/assessments/')

prev_sys_path = list(sys.path)
for directory in ALLDIRS:
    site.addsitedir(directory)

new_sys_path = []
for item in list(sys.path):
    if item not in prev_sys_path:
        new_sys_path.append(item)
        sys.path.remove(item)

sys.path[:0] = new_sys_path

os.environ['DJANGO_SETTINGS_MODULE'] = 'assessments-main.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

"The error message is pretty explicit already - mod_wsgi cannot find your settings file" I'm not sure about this. Note that the error messages asks "is there an error importing settings.py?". I take this to mean that _either_ it can't find it _or_ there's something wrong inside it! Further more, advice people give to debug this problem is to check whether python can import the settings file, implying this could be either. It it were more explicit that the problem is precisely that it can't find it, it would be more helpful. Are you sure that this is only about finding it? — GreenAsJade, Nov 20 '14 at 11:21
In fact, it _doesn't say "cannot find" it says "cannot import". This could be because it can't find it or because it fails to import, quite clearly. Which means the problem can be far and wide: permissions for access, paths in wsgi.py, or any number of reasons why apache can't import it while runserver can. Hence the need for a way of debugging what is going wrong :) — GreenAsJade, Nov 20 '14 at 11:28
Thanks for the pointer to IntegrationWithDjango, I can't believe I haven't seen that before! — GreenAsJade, Nov 20 '14 at 11:29

score 1 · Accepted Answer · answered Nov 28 '14 at 23:01

At last, an answer to the high level question: how to debug?

And it's so obvious:

print out everything that matters, from settings.py, and look in the apache log to see what you got, then compare it to what you get in development!

The linked article suggests this set of prints in settings.py:

import sys, os

print "__name__ =", __name__
print "__file__ =", __file__
print "os.getpid() =", os.getpid()
print "os.getcwd() =", os.getcwd()
print "os.curdir =", os.curdir
print "sys.path =", repr(sys.path)
print "sys.modules.keys() =", repr(sys.modules.keys())
print "sys.modules.has_key('mysite') =", sys.modules.has_key('mysite')
if sys.modules.has_key('mysite'):
  print "sys.modules['mysite'].__name__ =", sys.modules['mysite'].__name__
  print "sys.modules['mysite'].__file__ =", sys.modules['mysite'].__file__
  print "os.environ['DJANGO_SETTINGS_MODULE'] =", os.environ.get('DJANGO_SETTINGS_MODULE', None)

For my purposes, I also added DATABASE_URL, and later EMAIL_HOST_USER etc

The article also has a good explanation of how the whole startup process happens, which is worthwhile.

Note that it is a bit out of date with django itself, so the final "alternative wsgi.py" that he comes up with may not be so applicable, but the debugging technique certainly is.

Thanks for all the comments suggesting what my specific problem might be - they were helpful as well. — GreenAsJade, Nov 28 '14 at 23:02

score 0 · Answer 6 · answered Nov 15 '14 at 20:30

0

Can you try adding an empty __init__.py file in the cm_central directory? Depending on how the settings are being imported, this would be needed.

answered Nov 15 '14 at 20:30

vikramls

1,802
1
11
15

Django provides one of these - it's present. – GreenAsJade Nov 15 '14 at 22:55
Note that this "answer" is providing a suggestion about what might be wrong in my case (which I really do appreciate) but is not actually answering the question "how does one debug issues like this?". It really does seem that trial and error is the only way to tackle wsgi loading problems, which seems... suboptimal? – GreenAsJade Nov 15 '14 at 23:00

How to debug basic issues configuring django to be served with apache and mod-wsgi?

6 Answers6

Linked