How to specify in YAML to always create log file in the project's folder using dictConfig?

Question

In my Python program I have the following code:

def main():
    # The file's path
    path = os.path.dirname(os.path.realpath(__file__))
    ...
    # Config file relative to this file
    loggingConf = open('{0}/configs/logging.yml'.format(path), 'r')
    logging.config.dictConfig(yaml.load(loggingConf))
    loggingConf.close()
    logger = logging.getLogger(LOGGER)
    ...

and this is my logging.yml configuration file:

version: 1
formatters:
  default:
    format: '%(asctime)s %(levelname)s %(name)s %(message)s'
handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: default
    stream: ext://sys.stdout
  file:
    class : logging.FileHandler
    formatter: default
    filename: bot.log
loggers:
  cloaked_chatter:
    level: DEBUG
    handlers: [console, file]
    propagate: no

The problem is that the bot.log file is created where the program is launched. I want it to always be created in the project's folder, i.e. in the same folder as my Python program.

For an example, launching the program with ./bot.py would create the log file in the same folder. But launching it with python3 path/bot.py would create the log file a level above the Python program in the file hierarchy.

How should I write the filename in the config file to solve this? Or do I need to write a custom handler? If so, how? Or is this not possible to solve using dictConfig?

I updated my answer in response to your comment. – Vinay Sajip Apr 17 '13 at 14:43 — Vinay Sajip, Apr 17 '13 at 14:43

Vinay Sajip · Accepted Answer · 2013-04-17T14:43:10.510

There are a number of ways you can achieve what you want. For example, one way is to make a custom initialiser for your handler:

import os
import yaml

def logmaker():
    path = os.path.dirname(os.path.realpath(__file__))
    path = os.path.join(path, 'bot.log')
    return logging.FileHandler(path)

def main():
    # The file's path
    path = os.path.dirname(os.path.realpath(__file__))

    # Config file relative to this file
    loggingConf = open('{0}/logging.yml'.format(path), 'r')
    logging.config.dictConfig(yaml.load(loggingConf))
    loggingConf.close()
    logger = logging.getLogger('cloaked_chatter')
    logger.debug('Hello, world!')

if __name__ == '__main__':
    main()

Note that I moved the logging.yml to be adjacent to the script. The logmaker is the custom initialiser. Specify it in the YAML as follows:

version: 1
formatters:
  default:
    format: '%(asctime)s %(levelname)s %(name)s %(message)s'
handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: default
    stream: ext://sys.stdout
  file:
    () : __main__.logmaker
    formatter: default
loggers:
  cloaked_chatter:
    level: DEBUG
    handlers: [console, file]
    propagate: no

If you run the Python script, you should find that the bot.log is created adjacent to the script and YAML file. The same message is printed to the console and bot.log:

2013-04-16 11:08:11,178 DEBUG cloaked_chatter Hello, world!

N.B. The script could be a little tidier, but it illustrates my point.

Update: As per the documentation, the use of () as a key in dictionary indicates that the value is a callable which is essentially a custom constructor for the handler.

In the YAML code, what does `():` in front of `__main__.logmaker` mean? — Daniel Jonsson, Apr 17 '13 at 13:14
its the name of the "module" to invoke to get this data, for me I had to call my common module where i instantiate the logger @DanielJonsson — 4c74356b41, Jan 24 '19 at 13:01

How to specify in YAML to always create log file in the project's folder using dictConfig?

1 Answers1

Linked