17

I'm working on a page that has lots of images so this generates lots of output in the console of this type. In my dev environment I use django to serve static and media, so I get a LOT of this in my console:

...
[23/May/2014 12:41:54] "GET /static/css/style.css HTTP/1.1" 304 0
[23/May/2014 12:41:55] "GET /static/js/jquery-1.7.1.min.js HTTP/1.1" 304 0
[23/May/2014 12:41:55] "GET /static/js/jquery.form.js HTTP/1.1" 304 0
...
[23/May/2014 12:41:57] "GET /media/producto/Tapa_Santiago_Vazquez_SV.jpg HTTP/1.1" 304 0
[23/May/2014 12:41:57] "GET /media/CACHE/images/producto/Barcos_y_mariposas_DVD_baja/2e3e3894ca08f88c03459e00f9018427.jpg HTTP/1.1" 304 0
[23/May/2014 12:41:56] "GET /media/CACHE/images/producto/tapaDEJA_VU/fb67e92ffd47808a263db02ca016bc24.jpg HTTP/1.1" 304 0
...

making it very tedious to look for meaningful output.

I would like to filter out those messages in my environment so I only see the GET for the view and my output, but so far looking at the logging I saw that I could affect other logging from django but not this. I even tried this but it didn't work:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'handlers': {
        'null': {
            'level': 'ERROR',
            'class': 'django.utils.log.NullHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['null'],
            'level': 'ERROR',
            'propagate': True,
        },
    }
}

is it even possible to filter that kind of output out?

Thanks!!

Martin Massera
  • 1,718
  • 1
  • 21
  • 47

4 Answers4

23

Recent versions of Django make it really easy to override the default logging with your own LOGGING settings.

To filter out all GET requests to the /static/ directory, add the following to your settings.py:

def skip_static_requests(record):
    return not record.args[0].startswith('GET /static/')

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        # use Django's built in CallbackFilter to point to your filter 
        'skip_static_requests': {
            '()': 'django.utils.log.CallbackFilter',
            'callback': skip_static_requests
        }
    },
    'formatters': {
        # django's default formatter
        'django.server': {
            '()': 'django.utils.log.ServerFormatter',
            'format': '[%(server_time)s] %(message)s',
        }
    },
    'handlers': {
        # django's default handler...
        'django.server': {
            'level': 'INFO',
            'filters': ['skip_static_requests'],  # <- ...with one change
            'class': 'logging.StreamHandler',
            'formatter': 'django.server',
        },
    },
    'loggers': {
        # django's default logger
        'django.server': {
            'handlers': ['django.server'],
            'level': 'INFO',
            'propagate': False,
        },
    }
}

Here are all of Django's default loggers as of 1.10, which you can override in the same way: https://github.com/django/django/blob/32265361279b3316f5bce8efa71f2049409461e3/django/utils/log.py#L18

Here are descriptions of what Django's default built-in loggers do: https://docs.djangoproject.com/en/1.10/topics/logging/#id3

Here's are the docs on Django's CallbackFilter used above to hook in the custom filter: https://docs.djangoproject.com/en/1.10/topics/logging/#django.utils.log.CallbackFilter

alias51
  • 8,178
  • 22
  • 94
  • 166
tino
  • 4,780
  • 5
  • 24
  • 30
  • 3
    Nice solution! In my case, `record.args[0]` is sometimes a tuple (e.g. on "broken pipe" log messages), so I'm using the following lambda inline instead of your `skip_static_requests()` function: `lambda record: not isinstance(record.args[0], basestring) or not any([record.args[0].startswith('GET ' + prefix) for prefix in ['/static/']])`. – Henrik Heimbuerger Jul 04 '17 at 08:55
3

As a workaround, you can use this snippet (from Django Snippets):

from django.conf import settings
from django.core.servers import basehttp
from django.core.management.commands.runserver import Command as BaseCommand

class QuietWSGIRequestHandler(basehttp.WSGIRequestHandler):
    def log_message(self, format, *args):
        # Don't bother logging requests for paths under MEDIA_URL.
        if self.path.startswith(settings.MEDIA_URL):
            return
        # can't use super as base is old-style class, so call method explicitly
        return basehttp.WSGIRequestHandler.log_message(self, format, *args)

def run(addr, port, wsgi_handler):
    server_address = (addr, port)
    httpd = basehttp.WSGIServer(server_address, QuietWSGIRequestHandler)
    httpd.set_app(wsgi_handler)
    httpd.serve_forever()

class Command(BaseCommand):
    def handle(self, addrport='', *args, **options):
        # monkeypatch Django to use our quiet server
        basehttp.run = run
        return super(Command, self).handle(addrport, *args, **options)

You need to user this command to run the server. It basically only override the log behaviour to drop requests which begins by the MEDIA_URL setting. Put this whole file in an installed app and run it with ./manage.py runserver_quiet (if the command file is runserver_quiet.py)

You can't play with the logging module, because the WSGIRequestHandler doesn't use it to display these messages. It is directly written in the stderr stream.

Maxime Lorant
  • 34,607
  • 19
  • 87
  • 97
  • thanks! I will try it. This means, there is no way to do that with logging, right? wonder why! – Martin Massera May 27 '14 at 16:33
  • @MartinMassera Because if you look to the source code of the `WSGIRequestHandler` class, you will see the message is directly print into the `stderr` stream, sadly. https://github.com/django/django/blob/master/django/core/servers/basehttp.py#L87 I have added this explication into the answer. – Maxime Lorant May 27 '14 at 16:38
  • The code is not working for me, the "monkeypatch line" (basehttp.run = run) doesnt seem to be changing the run function. It is still using the original run function... – Martin Massera May 27 '14 at 19:27
  • I dont know how to create the wsgi_handler object to call run :) – Martin Massera May 27 '14 at 19:31
  • @MartinMassera Yeah I saw this after pressing "Add Comment". Does it use this command at least (put some print in `Command` and `log_message()`? – Maxime Lorant May 27 '14 at 19:32
  • yes I added debug prints in the `handle` and the `run` method, and also added in the django source code `run`, and the `handle` and django's `run` get called, but not the custom `run` – Martin Massera May 28 '14 at 19:43
  • Indeed, I also tried quickly. The `runserver` has changed since the creation of this snippet, so it might need to be adapt to work on Django 1.6. I think the [`run` import](https://github.com/django/django/blob/master/django/core/management/commands/runserver.py#L12) has something to do but I am really not sure. It need some tests but I don't have the time at the moment :( – Maxime Lorant May 28 '14 at 19:57
  • thank you I ll see if I can look at it too, its not that urgent for me... but I d like to have it correctly so I can upvote your answer – Martin Massera May 29 '14 at 03:21
2

From Django 1.10 you can configure django.server logging for "the handling of requests received by the server invoked by the runserver command". This is a quick way to filter out the media requests and to focus on your logger info.

Add to django logging settings:

loggers: {
....
    'django.server': {
        'handlers': ['console'],
        'level': 'ERROR'  # or INFO if you want to see the log
    },
}

Django Documentation ref: Logging: django-server

Tim
  • 21
  • 2
0
from django.core.servers.basehttp import WSGIRequestHandler

# Grab the original log_message method.
_log_message = WSGIRequestHandler.log_message

def log_message(self, *args):
    # Don't log if path starts with /static/
    if self.path.startswith("/static/"):
        return
    else:
        return _log_message(self, *args)

# Replace log_message with our custom one.
WSGIRequestHandler.log_message = log_message

# Import the original runserver management command
from django.core.management.commands.runserver import Command

Hat tip https://code.djangoproject.com/ticket/25704

Neil
  • 8,925
  • 10
  • 44
  • 49