How to enable daily new filename of logging in Python Django's Settings?

Question

I want to log the calling of restful api on each day into yyyy-mm-dd.log. In settings.py I write:

LOGGING = {
'version': 1,
'disable_existing_loggers': False,  
'formatters': {
    'standard': {
        'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
    },
},
'handlers': {
    'infofile': {
        'level': 'INFO',
        'class': 'logging.FileHandler',
        'filename': 'C://mydoc/logs/'+str(datetime.date.today())+'.log',
        'formatter':'standard',
    },        
},
'loggers': {      
    'django': {
        'handlers': ['infofile'],
        'propagate': True,
    },        
},

If the server started e.g. on 01.01.2018, the "2018-01-01.log" will be created and all logs will be written into the file, even the callings is on 15.01, because the settings.py is run only one time on 01.01 at the server's starting.

How can Django create everyday a new log file and use the file?

trinchet · Accepted Answer · 2018-01-15T20:09:40.647

4

This can be easily implemented using TimedRotatingFileHandler

...
'handlers': {
    'infofile': {
        'level': 'INFO',
        'class': 'logging.handlers.TimedRotatingFileHandler',
        'filename': 'yourlogbasename.log',
        'when': 'D', # daily, you can use 'midnight' as well
        'backupCount': 100, # 100 days backup
        'formatter': 'verbose',
    },
}
...

See more details at Django log to file by date or hour

Alternatively, you can use an external tool to take care of this, e.g logrotate See daily as option in the configuration of this tool to fit your need.

edited Jan 15 '18 at 20:09

answered Jan 15 '18 at 19:49

trinchet

6,753
4
37
60

1

I have now "PermissionError: [WinError 32] The process cannot access the file because it is being used by another." So the logfile cannot be renamed. why? – Üh Lieh Jan 16 '18 at 16:41
Can you post the entire traceback? In any case, make sure the process that is running django has permission to write on `C://mydoc/logs/` – trinchet Jan 16 '18 at 22:05
traceback is below – Üh Lieh Jan 17 '18 at 11:52
@ÜhLieh: see https://stackoverflow.com/a/26848236 about the `PermissionError` – djvg Mar 09 '20 at 13:12

score 0 · Answer 2 · answered Jan 15 '18 at 17:26

One way you can do it would be to use a daemon process within your django app that runs every X days and makes new log file for you to write to.

A library I would recommend is APScheduler which allows you to decorate functions to be run on a crontab like schedule.

score 0 · Answer 3 · edited Jan 17 '18 at 14:08

I have now "PermissionError: [WinError 32] The process cannot access the file because it is being used by another." So the logfile cannot be renamed. why?

Traceback (most recent call last):

  File "C:\Users\E945826\Python\lib\logging\handlers.py", line 72, in emit
    self.doRollover()

  File "C:\Users\E945826\Python\lib\logging\handlers.py", line 396, in doRollover

    self.rotate(self.baseFilename, dfn)

  File "C:\Users\E945826\Python\lib\logging\handlers.py", line 113, in rotate
    os.rename(source, dest)

PermissionError: [WinError 32] The process cannot access the file because it is being used by another: 'C:\\logs\\apicalls' -> 'C:\\logs\\apicalls.2018-01-17_11-01'

Call stack:

  File "C:\Users\E945826\Python\lib\threading.py", line 884, in _bootstrap

    self._bootstrap_inner()

  File "C:\Users\E945826\Python\lib\threading.py", line 916, in _bootstrap_inner

    self.run()

  File "C:\Users\E945826\Python\lib\threading.py", line 864, in run

    self._target(*self._args, **self._kwargs)

  File "C:\Users\E945826\Python\lib\socketserver.py", line 639, in process_request_thread

    self.finish_request(request, client_address)

  File "C:\Users\E945826\Python\lib\socketserver.py", line 361, in finish_request

    self.RequestHandlerClass(request, client_address, self)

  File "C:\Users\E945826\Python\lib\socketserver.py", line 696, in __init__

    self.handle()

  File "C:\Users\E945826\Python\lib\site-packages\django\core\servers\basehttp.py", line 155, in handle

    handler.run(self.server.get_app())

  File "C:\Users\E945826\Python\lib\wsgiref\handlers.py", line 138, in run

    self.finish_response()

  File "C:\Users\E945826\Python\lib\wsgiref\handlers.py", line 183, in finish_response

    self.close()

  File "C:\Users\E945826\scrubbing

Python\lib\wsgiref\simple_server.py", line 35, in close

    self.status.split(' ',1)[0], self.bytes_sent

  File "C:\Users\E945826\Python\lib\http\server.py", line 536, in log_request

    self.requestline, str(code), str(size))

  File "C:\Users\E945826\Python\lib\site-packages\django\core\servers\basehttp.py", line 124, in log_message

    level(format, *args, extra=extra)

Try to run your dev server using: `manage.py runserver --noreload`. Also, please remove this (your) answer when you are done, this is not actually an answer to the original question. — trinchet, Jan 17 '18 at 20:25

How to enable daily new filename of logging in Python Django's Settings?

3 Answers3