Problems debugging Django `collectstatic` in a cloudControl deployment

Question

I have a Django application deployed to cloudControl. Configuration is standard and the push/deploy happens without (apparent) errors.

But the collectstatic step is not being executed: it fails silently (I see no -----> Collecting static files message). After the deploy, the static folder for the application is empty, so you get 500 Server Errors continually.

I can solve it changing the Procfile, but it is not consistent either:

web: python manage.py collectstatic --noinput; gunicorn app.wsgi:application --config gunicorn_cnf.py --bind 0.0.0.0:${PORT:-5000}`

collectstatic works as it should locally, and if I run cctrlapp app/deployment run "python manage.py collectstatic --noinput" no errors are shown either:

669 static files copied to '/srv/www/staticfiles/static', 669 post-processed.

But /srv/www/staticfiles/static is empty.

How can I know why collectstatic is not being executed in the push phase?

FYI: the `staticfiles/static` folder is included in the repo, with a dummy file. The deployment has the folder created as expected... with no dummy file (a `.gitignore`). — eillarra, Mar 31 '15 at 13:25
Have you tried inspecting the run-time environment with `run bash` for potential path problems that resulted in the buildpack output and the actual filesystem structure to differ? Unlikely, but just to rule that out, the output directory is not by any chance included in a `.cctrlignore` file? https://www.cloudcontrol.com/dev-center/platform-documentation#image-building — pst, Apr 01 '15 at 12:23
No, no `.cctrlignore` file is used. If I `run bash` I can see the the static files folder where it should be (`/srv/www/staticfiles/static`) but it is empty. If I `run "python manage.py collectstatic --noinput"` a success message is shown, but files are not added either (I guess because workers are different). The only way of having the files is to prepend the command to the `Procfile`, but then I get some inconsistencies (not all statics are collected: for example for the django-pagedown package, that relies on a git module). — eillarra, Apr 01 '15 at 14:30
@pst View answer with my conclusions. Any other cleaner way of checking if we are in the push phase? — eillarra, Apr 03 '15 at 09:49
There is a environment variable `BUILDPACK_RUNNING` set while the buildpack is running during push. Impressive work, debugging this issue. — pst, Apr 03 '15 at 12:32
@pst Thanks! I have edited my answer adding a reference to `BUILDPACK_RUNNING`. — eillarra, Apr 03 '15 at 14:05

eillarra · Accepted Answer · 2015-04-04T06:45:48.733

I've been able to debug the problem, using a custom python buildpack, so here is the answer for further reference.

The problem was in the settings.py file. The first thing I do in this file is to check if we are in a cloudControl environment or in a local environment. I do it looking for the CRED_FILE environment variable (not so different of what is suggested): if no variable is found, I load a local JSON file that mimics that credentials variable for development:

try:
    cred_file = os.environ['CRED_FILE']
    DEBUG = False

except KeyError:
    cred_file = os.path.join(BASE_DIR, 'creds.json')
    DEBUG = True

Once I know the environment, I can have different INSTALLED_APPS (requirements.txt files are slightly different in production and development, too) or change some settings.

Now the bad news: in the push phase there is no CRED_FILE available.

So I was trying to load apps that were not installed (because they were only in the development requirements file, like coverage or django-debug-toolbar) or use credentials that were not set (creds.json is, of course, not uploaded to the repository: only a TXT with dummy values is uploaded as a reference). That's why collectstatic was failing silently in the push phase.

Here is my solution (it will work as long as you have a dummy credentials file in your repo):

try:
    cred_file = os.environ['CRED_FILE']
    DEBUG = False

except KeyError:
    if os.path.exists(os.path.join(BASE_DIR, 'creds.json')):
        cred_file = os.path.join(BASE_DIR, 'creds.json')
        DEBUG = True
    else:
        cred_file = os.path.join(BASE_DIR, 'creds.json.txt')
        DEBUG = False

Credentials are not used by collectstatic, so you can have anything in the creds.json.txt file. Not very clean, but it works as expected now.

EDIT

As pointed by @pst in a comment there is an environment variable to know if the buildpack is running, so we could use that one too to load the desired credentials and set DEBUG.

if 'CRED_FILE' in os.environ:
    cred_file = os.environ['CRED_FILE']
    DEBUG = False

elif 'BUILDPACK_RUNNING' in os.environ:
    cred_file = os.path.join(BASE_DIR, 'creds.json.txt') 
    DEBUG = False

else:
    cred_file = os.path.join(BASE_DIR, 'creds.json')
    DEBUG = True

The way my settings are configured yes. But it is not a problem because the dummy file is always in the repo as a reference for developers (they need to fill in the right settings for local development and rename it). — eillarra, Apr 03 '15 at 19:52
@pst I could remove dummy file if I then use `dict.get('key', 'default_cred')` when loading the credentials. What I do need if I use this way is to check if `cred_file`is defined using a `except NameError` and set a couple of empty dictionaries... — eillarra, Apr 04 '15 at 06:49

Problems debugging Django `collectstatic` in a cloudControl deployment

1 Answers1