supervisord logs don't show my output

Question

I have a [program:x] running and it prints / sys.stdout.writes a lot of things. None of which comes up in either in the AUTO childlogdir of [supervisord] or in stdout_logfile of [program:x] Am I missing something?

How do I capture all that is printed or stdout-ed from [program:x] ?

In my program I am explicitly doing both,

print "something"
sys.stdout.write("something")

Relevant supervisord.conf file

[supervisord]
childlogdir = %(here)s/../logs/supervisord/
logfile = %(here)s/../logs/supervisord/supervisord.log
logfile_maxbytes = 100MB
logfile_backups = 10
loglevel = info
pidfile = %(here)s/../logs/supervisord/supervisord.pid
umask = 022
nodaemon = false
nocleanup = false

[program:x]
directory = %(here)s/../
command = python file.py
autostart=true
autorestart=true
redirect_stderr=true  
stdout_logfile = /appropriate-path/to/access.log

Are you certain the program prints to `stdout` and not `stderr`? — Martijn Pieters, Dec 18 '12 at 14:17
edited the question with more info. When I do a sys.stdout.write, it should be printing to stdout — zubinmehta, Dec 18 '12 at 14:36
And what is your supervisord config for that `[program:..]`? Oh, and try a `sys.stdout.flush()` as well to make sure it's not the buffer. — Martijn Pieters, Dec 18 '12 at 14:40
I put in the entire conf file. Also, I tried putting a `sys.stdout.flush()` at the end of my file.py but no luck. — zubinmehta, Dec 18 '12 at 14:51
If you check your supervisord log file, do you actually see that your process was started? If the program never ran, it cannot output anything. — Mark Hildreth, Dec 18 '12 at 15:00
What version of supervisord? I cannot reproduce your problem locally with 3.0a10; it *could* be 3.0b1 broke something, but let's see your version number first. — Martijn Pieters, Dec 18 '12 at 15:04
Yes my version is supervisor==3.0b1 I will go back to a stabler one, rerun and notify back. — zubinmehta, Dec 18 '12 at 15:07
I am now at supervisor==3.0a12 but no luck. So, nothing to do with supervisord version I suppose. — zubinmehta, Dec 18 '12 at 15:38
3.0b1 *is* the most current version. `a` in 3.0a12 stands for `alpha`, which usually comes before `beta`. :-) We've used supervisord 3.0 alpha releases in production for years now. — Martijn Pieters, Dec 18 '12 at 21:45
ohh I got it the other way round! But anyhow, the problem is somewhere else and not in supervisord 3.0b1 is what I feel. Thanks for all the help. Will try to do some more trials to fix this or will go some other route. — zubinmehta, Dec 19 '12 at 07:48
@MartijnPieters sys.stdout.flush() helped. I was doing it in the wrong place earlier. you can post that as the answer. thanks. — zubinmehta, Dec 19 '12 at 15:50
This is happening to me on a PHP command-line app. It logs fine for a while, then suddenly just stops logging, even though the app continues to run as expected. This is HIGHLY inconvenient behavior for this particular app, and my attempts to ob_flush() and flush() have not improved the issue. WTF is going on... — Brade, Jan 19 '15 at 15:13

Martijn Pieters · Accepted Answer · 2019-02-27T14:16:41.523

Python output is buffered. Setting the environment variable PYTHONUNBUFFERED=1 in you supervisord.conf will disable buffering and show log messages sooner:

[program:x]
environment = PYTHONUNBUFFERED=1

or add the -u command-line switch to python command:

[program:x]
command = python -u file.py

Alternatively you can flush the sys.stdout handler explicitly:

sys.stdout.flush()

On python 3.3 and up, you can add the flush=True parameter to have the function do this for you:

print(something, flush=True)

score 48 · Answer 2 · answered Jul 31 '13 at 04:19

48

You can run your program like this:

python -u file.py

this will produce unbuffered output

answered Jul 31 '13 at 04:19

Dmitry Belaventsev

6,347
12
52
75

1

Proper (15char) – Tim Mar 31 '17 at 18:47

score 1 · Answer 3 · answered Oct 13 '17 at 12:05

If you have python based script that you can't or don't want to change to flush output on a regular base then you can use unbuffer from the Expect package.

For a Django application in a docker container I've lately used it like that (from a shell script run from supervisord):

unbuffer python -u manage.py runserver 0.0.0.0:11000 2>&1 >> /var/log/django.log

supervisord logs don't show my output

3 Answers3

Linked