37

I want to configure supervisor to control gunicorn in my django 1.6 project using an environment variable for SECRET_KEY.

I set my secret key in .bashrc as 

export SECRET_KEY=[my_secret_key]

And I have a shell script to start gunicorn:

NAME="myproject"
LOGFILE=/home/django/myproject/log/gunicorn.log
LOGDIR=$(dirname $LOGFILE)
NUM_WORKERS=3
DJANGO_WSGI_MODULE=myproject.wsgi

USER=django
GROUP=django
IP=0.0.0.0
PORT=8001

echo "Starting $NAME"

cd /home/django/myproject/myproject
source /home/django/.virtualenvs/myproject/bin/activate

test -d $LOGDIR || mkdir -p $LOGDIR

exec gunicorn ${DJANGO_WSGI_MODULE} \
 --name $NAME \
 --workers $NUM_WORKERS \
 --user=$USER --group=$GROUP \
 --log-level=debug \
 --bind=$IP:$PORT
 --log-file=$LOGFILE 2>>$LOGFILE

Then to configure my project's gunicorn server in supervisor:

[program:my_django_project]
directory=/home/django/my_django_project/my_django_project
command=/home/django/my_django_project/my_django_project/gunicorn.sh
user=django
autostart=true
autorestart=true
stdout_logfile=/home/django/my_django_project/log/supervisord.log
stderr_logfile=/home/django/my_django_project/log/supervisor_error.log

If I start gunicorn using my shell script it doesn't throw any error but when I start it with supervisor it fails and I see in the logs that it doesn't "find" my SECRET_KEY.

What's the correct way to configure supervisor to read my shell variables (I wan't to keep them in my .bashrc unless there's a more appropriate way)?

equalium
  • 1,241
  • 2
  • 12
  • 18
  • 1
    A process's environment is readable by other users on the system so this is completely insecure. It would be much more secure to store the key in a file only readable by the user that the process runs as. – AutomatedMike May 24 '16 at 08:00

4 Answers4

28

OK, I guess I got it.

I had tried including

environment=SECRET_KEY="secret_key_with_non_alphanumeric_chars"

in the conf file for supervisor but it didn't like the non alphanumeric chars and I didn't want to have my key in the conf file as I have it in git.

After loking at supervisor's docs I had also tried with:

HOME="/home/django", USER="django"

but didn't work.

Finally I tried with this and is working now!:

environment=HOME="/home/django", USER="django", SECRET_KEY=$SECRET_KEY

Maybe although it's working it's not the best solution. I'd be happy to learn more.

EDIT:

Finally, Ewan made me see that using the bash for setting the env vars wouldn't be the best option. So one solution, as pointed by #Ewan, would be to use:

[program:my_project]
...
environment=SECRET_KEY="secret_key_avoiding_%_chars"

Another solution I found, for those using virtualenv would be to export the env vars in the "activate" script of the virtualenv, that is, edit your virtualenv/bin/activate file and add at the end your SECRET_KEY.

This way you can use % chars as generated by key generators for django and is valid if you don't use supervisor.

I restarted my server without logging to check that it worked. With this option I don't have to edit my keys, I can keep my conf files versioned and it works whether I use supervisor, upstart or whatever (or nothing, just gunicorn).

Anyway, I know I haven't discovered anything new (well @Ewan raised an issue with supervisor) but I'm learning things and hope this can be useful to someone else.

Alp
  • 29,274
  • 27
  • 120
  • 198
equalium
  • 1,241
  • 2
  • 12
  • 18
  • Clarification: I have my conf file versioned because I use the ubuntu package that stores the configuration as separate files in /etc/supervisor/conf.d/ and I'm going to have various projects in the same machine with their respective project.conf files. – equalium Sep 27 '13 at 15:28
  • 3
    I put my Environmental Variables in the `../bin/activate` virtualenv script and that worked for me. thnx – Aaron Lelevier Sep 20 '14 at 14:45
  • 1
    This `environment=HOME="/home/django", USER="django", SECRET_KEY=$SECRET_KEY` won't work with newer versions of Supervisor. From their github: _Supervisor 3.2.0 (November 30, 2015) supports using %(ENV_x)s expansions in environment=_ – BringBackCommodore64 Mar 17 '16 at 14:07
11

Also if you use gunicorn config file:

gunicorn -c gunicorn.py myproject.wsgi

It's possible to pass environment variables in the gunicorn.py file like this:

bind = "0.0.0.0:8001"
workers = 3
proc_name = "myproject"
user = "django"
group = "django"
loglevel = "debug"
errorlog = "/home/django/myproject/log/gunicorn.log"
raw_env = [
   'DATABASE_URL=postgres://user:password@host/dbname',
   'SECRET_KEY=mysecretkey',
]
MadisonTrash
  • 5,444
  • 3
  • 22
  • 25
  • Wouldn't this defeat the purpose of using supervisor and gunicorn together? Wouldn't supervisor spawn different django projects from one gunicorn file? If that is the case then the solution you propose will work for only one project. – BringBackCommodore64 Mar 17 '16 at 14:14
  • 1
    @MadisonTrash I realise this is a bit old now but did you ever try that? I'm trying that now and I'm not seeing any of the `raw_env` I've set make it through into the `/proc//environ` of gunicorn processes (neither the root process or the spawned processes). – Samuel Harmer Aug 20 '18 at 22:26
6

Your .bashrc will only work for interactive shells so will work when running the shell script as your user however supervisor, running in the background, wont get passed these values.

Instead, in your supervsior .ini file set the environment variable there (more information in the documentation).

e.g.

[program:my_django_project]
environment=SECRET_KEY="my_secret_key"

After a little bit of trial and error, I noticed that the supervisor .ini file doesn't like to have % in the environment variables section (even if you do quote it...). Based on your example in the comments I have tried this with supervisor==3.0 installed via pip and it works:

environment=SECRET_KEY="*wi4h$kqxp84f3w6uh8w@l$0(+@x$3cr&)z^lmg+pqw^6wkyi"

Only difference is I have removed the% sign. (I tried escaping it with \% but this still didn't work)

Edit 2

Raised issue #291 with supervisor for this bug.

Edit 3

As noted in above issue, if a % is present in your secret key it must be escaped python-style: %%

Community
  • 1
  • 1
Ewan
  • 14,592
  • 6
  • 48
  • 62
  • Thanks Ewan, I found and posted a solution to use my .bashrc. Do you know if what I'm doing is conceptually wrong? – equalium Sep 27 '13 at 15:31
  • @equalium - you may have a problem if your server reboots without you logging in. It may not read your `.bashrc` therefore not set your `$SECRET_KEY` causing your app to fail. For clarification read [this](http://superuser.com/questions/183870/difference-between-bashrc-and-bash-profile). You can also try setting the environment variables at the [Nginx level](http://serverfault.com/questions/351696/how-do-you-set-server-specific-env-values-in-nginx) (I don't know what your non-alphanumeric values are I'm afraid so can't test myself) – Ewan Sep 27 '13 at 15:52
  • That makes sense! I should set a bulletproof system. My secret key is the type of "*wi4h$kqxp84f3w6uh8w@l$0(+@x$3cr&)z^lmg+pqw^6w%kyi". If I do sudo supervisorctl reread I get: CANT REREAD: Format string 'SECRET_KEY="..."' for environment is badly formatted. – equalium Sep 27 '13 at 16:06
  • Thanks again @Ewan, great job. I'm new to supervisor and I'm starting to learn quickly. I should keep the conf files out of version control and set my env vars with the 'environment section' avoiding the % char. – equalium Sep 27 '13 at 21:20
1

You can escape % character by adding another % character.

Otherwise, quoting the values is optional but recommended. To escape percent characters, simply use two. (e.g. URI="/first%%20name")

Taken from here: http://supervisord.org/configuration.html

Simon Tremblay
  • 51
  • 1
  • 4