Unable to name log file with custom timestamp format using TimedRotatingFileHandler

Question

Is it possible to create log files with custom date format, using TimedRotatingFileHandler?

import logging
from logging.handlers import TimedRotatingFileHandler
import os
from datetime import datetime
import time

logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)

log_dir = 'logs'
log_file = os.path.join(log_dir, datetime.now().strftime('%Y%m%d-%H%M%S') + f'_.log')

handler = TimedRotatingFileHandler(log_file, when='s', interval=1, backupCount=0)
handler.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s'))
logger.addHandler(handler)

i = 0
while True:
    logger.info(i)
    print(i)
    time.sleep(1)
    i += 1

Produces

20230810-100547_.log.2023-08-10_10-05-47
20230810-100547_.log.2023-08-10_10-05-48
20230810-100547_.log

Instead of

20230810-100547_.log
20230810-100548_.log
20230810-100549_.log

Does one need to reset the handler after midnight (is it even possible)? Does not seem elegant.

There seem to be a namer attribute in logging handlers

score 1 · Accepted Answer · answered Aug 10 '23 at 08:31

1

There is rotation_filename method and you can assign it to namer function. It will change name of your log file during rotation. Working code should look like this:

import logging
from logging.handlers import TimedRotatingFileHandler
import os
from datetime import datetime
import time

def namer(self):
    now = datetime.now()
    log_dir = './logs'
    return os.path.join(log_dir, datetime.now().strftime('%Y%m%d-%H%M%S') + f'_.log')

logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)

log_dir = './logs'
log_file = os.path.join(log_dir, datetime.now().strftime('%Y%m%d-%H%M%S') + f'_.log')

handler = TimedRotatingFileHandler(log_file, when='s', interval=1, backupCount=0)
handler.rotation_filename = namer
handler.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s'))
logger.addHandler(handler)


i = 0
while True:
    logger.info(i)
    print(i)
    time.sleep(1)
    i += 1

answered Aug 10 '23 at 08:31

jlipinski

152
12

Thanks - this is a big help! One question: every time an interval is reached, it renames existing log file and creates new log file one conforming to initial `log_file` string. Could it avoid the renaming step? With daily log, you know after midnight the final name of log (`YYYYMMDD.log`). – learning_python_self Aug 10 '23 at 08:50
1

To my knowledge - no. As far I understand Rotating loggers in python, they work like this: 1. Create a "log" file 2. When interval is reached rename existing "log" file to e.g. "log.1" file and create new "log" file. Maybe there is a workaround by using non rotating logger and creating new handler every interval, but that doesn't seem right. – jlipinski Aug 10 '23 at 08:52
I am now wondering if with midnight interval, it will name the log for yesterday with today's timestamp. I guess one can remove 1 day in `namer` function. Thanks again! – learning_python_self Aug 10 '23 at 08:54
1

@jlipinski you have it almost right, but not quite You shouldn't monkey-patch the rotation_filename method, but set the namer attribute - `handler.namer = custom_namer_func` - see https://docs.python.org/3/library/logging.handlers.html#logging.handlers.BaseRotatingHandler.namer – Vinay Sajip Aug 10 '23 at 18:20

Unable to name log file with custom timestamp format using TimedRotatingFileHandler

1 Answers1