How to create a new log file every time the application runs using logger.config?

Question

how i can create new info.log file each time app runs. Let me know the cahnges i need to make in this config. I know we have to use doRollover or rotate but how to implement in this config not getting any idea. Any help will be appreciated and thanks in advance.

file-1. <<log.py >>

import configparser
import logging.config

code_dir_path = os.path.abspath(os.path.dirname(__file__))

log = logging.getLogger(__name__)

logging_conf = \
    {u'disable_existing_loggers': False,
     u'formatters': {u'basic': {
         u'format': u'%(asctime)s - %(name)s - %(threadName)s : %(levelname)s : %(module)s : %(lineno)d : %(message)s'}},
     u'handlers': {u'console': {u'class': u'logging.StreamHandler',
                                u'formatter': u'basic',
                                u'level': u'DEBUG',
                                u'stream': u'ext://sys.stdout'},
                   u'info_file_handler': {u'backupCount': 20,
                                          u'class': u'logging.handlers.RotatingFileHandler',
                                          u'encoding': u'utf8',
                                          u'filename': u'info.log',
                                          u'formatter': u'basic',
                                          u'level': u'INFO',
                                          u'maxBytes': 10485760}},
     u'loggers': {u'APP_LOG': {u'handlers': [u'console'],
                               u'level': u'INFO',
                               u'propagate': u'no'}},
     u'root': {u'handlers': [u'console', u'info_file_handler'], u'level': u'INFO'},
     u'version': 1}


def setup_logging(json_filename='logging_conf.json', default_level=logging.INFO,
                  env_key='LOG_CFG', log_file_basepath=".", customised_file_name=""):
    """
      Setup logging configuration
    """
    json_filepath = os.path.join(log_file_basepath, json_filename)
    env_json_filepath = os.getenv(env_key, None)
    if env_json_filepath:
        json_filepath = env_json_filepath
    if logging_conf:
        log_file_name = logging_conf["handlers"]["info_file_handler"]["filename"]
        if customised_file_name:
            log_file_name = customised_file_name
        logging_conf["handlers"]["info_file_handler"]["filename"] = os.path.join(log_file_basepath, log_file_name)
        logging.config.dictConfig(logging_conf)
    else:
        logging.basicConfig(level=default_level)

file-2. <<runner.py>>

import logging
module_logger = logging.getLogger('JIRA.fetch_data')
setup_logging()
module_logger.info("**Start capturing logs using logging.config")

Answer 1

There are a couple ways to do this. The first is to subclass logging.handlers.RotatingFileHandler and use the subclass:

file 1: <log.py>

class CustomRotatingFileHandler(logging.handlers.RotatingFileHandler):
    """Rollover when logging gets configured."""

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        print('rollover')
        self.doRollover()
        

LOGGING = {
    'version': 1,
    'handlers': {
        'logfile': {
            '()': CustomRotatingFileHandler,  # <<< not in quotes
            'filename': 'test.log',
            'backupCount': 3,
        }
    },
    'root': {
        'level': 'DEBUG',
        'handlers': ['logfile']
    },
}

logging.config.dictConfig(LOGGING)

The '()' key tells the logging configuration code that you are using custom code to create the handler. If the value is a callable (e.g., a class constructor), it is called with the rest of the dict items ('filename' and 'backupcount') passed as keyword arguments.

A second way to do it is to use a function instead of a subclass:

def force_rollover(wrapped_class):
    def rollover(*args, **kwargs):
        instance =  wrapped_class(*args, **kwargs)
        instance.doRollover()
        return instance
    return rollover

LOGGING = {
    'version': 1,
    'handlers': {
        'logfile': {
            '()': force_rollover(logging.handlers.RotatingFileHandler),
            'filename': 'info_log',
            'backupCount': 3}
    },
    'root': {
        'level': 'DEBUG',
        'handlers': ['logfile']
    },
}

logging.config.dictConfig(LOGGING)

The function force_rollover() takes a subclass of BaseRotatingHandler and returns a new function. The '()' key in the config dict causes the new function to be called, which creates an instance of the subclass, calls its doRollover() method, and then returns the instance. The returned instance is then used as the handler.

A third approach would be to get the root logger and use the getChild() method to recursively search all the loggers and check their handlers attribute to find the handler you want to rollover. This seems like more work than the first two approaches, so I won't go into it.