Python: initiated Logger with dataclass field param

Question

This is my Logger class:

import logging
import os
import datetime


class Logger:
    _logger = None

    def __new__(cls, user: str, *args, **kwargs):
        if cls._logger is None:
            cls._logger = super().__new__(cls, *args, **kwargs)
            cls._logger = logging.getLogger("crumbs")
            cls._logger.setLevel(logging.DEBUG)
            formatter = logging.Formatter('[%(asctime)s] [%(levelname)s] [%(filename)s] [%(funcName)s] [%(lineno)d]: %(message)s')
            now = datetime.datetime.now()
            directory_name = f'./log/{now.strftime("%Y-%m-%d")}'
            base_log_name = f'/{user}'
            log_file_extension = '.log'
            if not os.path.isdir(directory_name):
                os.mkdir(directory_name)
            file_handler = logging.FileHandler(f'{directory_name}{base_log_name}{now.strftime("%d-%m-%Y")}{log_file_extension}', 'w', 'utf-8')
            stream_handler = logging.StreamHandler()
            file_handler.setFormatter(formatter)
            stream_handler.setFormatter(formatter)
            cls._logger.addHandler(file_handler)
            cls._logger.addHandler(stream_handler)
        return cls._logger

And this is my class that accept user argument and I want my log file to be created with my user name in the file name:

@dataclass(kw_only=True)
class RunningJobManager:
    user: str = field(init=True)
    password: str = field(init=True)
    logging: Logger = field(init=False, default_factory=Logger(user=user))

So currently my user field inside Logger class is with type of dataclasses.Field instead of string. I also try to use default instread of default_factory

And I got this error:

Curerenly my code crash with OSError, [Errno 22] Invalid argument: 'G:\my_project\log\2023-01-20\Field(name=None,type=None,default=<dataclasses._MISSING_TYPE object at 0x0000017B1E78DB10>,default_factory=<dataclasses._MISSING_TYPE object at 0x0000017B1E78DB10>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=<dataclasses._MISSING_TYPE object at 0x0000017B1E78DB10>,_field_type=None)20-01-2023.log'

At this line:

file_handler = logging.FileHandler(f'{directory_name}{base_log_name}{now.strftime("%d-%m-%Y")}{log_file_extension}', 'w', 'utf-8')

EDIT

"stdout_handler": {
                    "formatter": "std_out",
                    "class": "logging.StreamHandler",
                    "level": "DEBUG"
                }

could you explain what problem are you trying to solve in details? — Danila Ganchar, Jan 20 '23 at 11:30
ok. I understand. But what is the main problem? As I understood you need specific logger per some `JobManager` after initialization. Why you don't want to use [dictConfig](https://docs.python.org/3/library/logging.config.html#logging.config.dictConfig) inside `constructor` for example? — Danila Ganchar, Jan 20 '23 at 11:43
I don't familiar with dictConfig, Whats the advantages of dictConfig ? — falukky, Jan 20 '23 at 11:51
As of the current documentation, `default_factory` takes a zero-argument-callable, so you cannot pass it `user=...`. Why not use a regular class? — CodeCop, Jan 20 '23 at 11:55
@falukky you don't need `setFormatter`, `addHandler` etc. you just initialize all loggers from a config file. I can show an example if you need. — Danila Ganchar, Jan 20 '23 at 11:58

score 1 · Accepted Answer · answered Jan 20 '23 at 12:28

Here is example with dictConfig:

import logging
import os
from dataclasses import dataclass, field
from logging import Logger
from logging.config import dictConfig


@dataclass(kw_only=True)
class AbstractJobManager:
    user: str = field(init=True)

    @property
    def logger(self) -> Logger:
        return logging.getLogger('crumbs')

    def __post_init__(self):
        if not os.path.isdir('log'):
            os.mkdir('log')

        # some config from YAML, TOML or hardcoded - doesn't matter...
        config = {
            'version': 1,
            'loggers': {
                'crumbs': {
                    'level': 'INFO',
                    'handlers': ['file_handler'],
                },
            },
            'handlers': {
                'file_handler': {
                    'level': 'INFO',
                    'class': 'logging.handlers.RotatingFileHandler',
                    'filename': f'./log/{self.user}.log',  # I skipped %Y-%m-%d...
                    'backupCount': 3,
                    'formatter': 'simple',
                },
            },
            'formatters': {
                'simple': {
                    'format': '%(pathname)s:%(lineno)d %(asctime)s - %(levelname)s - %(message)s',
                },
            },
        }

        dictConfig(config)


@dataclass(kw_only=True)
class RunningJobManager(AbstractJobManager):
    password: str = field(init=True)

    def run(self):
        self.logger.info('user = %s, password = %s', self.user, self.password)


RunningJobManager(user='falukky', password='Hello World!').run()
RunningJobManager(user='ridley', password='scott').run()
RunningJobManager(user='jimi', password='hendrix').run()

@falukky the main difference is that you program an initialization and configuration. In case of `dictCofig`, you have a logging configuration in one line `dictConfig(config)`. Without additional classes and code. This way you can put all settings in any readable format. Let's say you need to change `level`, add / remove `handler`, disable a `logger` etc. You do this without modifying the application code - you can just change an option in config file and restart application. — Danila Ganchar, Jan 24 '23 at 00:43
I have a question: With your code whenI start my script using cmd windows I cannot see the log output on the screen but only at the end in the .log file that created, Any idea why ? — falukky, Jan 30 '23 at 19:57
@falukky because I only have `file_handler` in my example. You just need to add `StreamHandler` in config(example: `{'handlers': {'stdout_handler': {'level': 'DEBUG', 'class': 'logging.StreamHandler'...`) and add `stdout_handler` to `crumbs.handlers`. — Danila Ganchar, Jan 30 '23 at 21:09
Please see my update, I add it to my handlers and got this error: ValueError: Unable to configure handler 'stdout_handler' — falukky, Feb 01 '23 at 16:12
@falukky you using `std_out` formatter. Did you registered it? You should if no. JFYI: You can use one formatter for all handlers(`"formatter": "simple"`). — Danila Ganchar, Feb 02 '23 at 08:38

score 0 · Answer 2 · answered Jan 20 '23 at 12:03

This is not the way dataclasses handle initialization of a field depending on other fields. The default_factory parameter of field is expected to be a zero-argument callable. The standard library documentation states:

default_factory: If provided, it must be a zero-argument callable that will be called when a default value is needed for this field...

Said differently it is not possible to pass any argument to default_factory.

The correct way to initialize a field using values of other fields is to use a __post_install__ method:

@dataclass(kw_only=True)
class RunningJobManager:
    user: str = field(init=True)
    password: str = field(init=True)
    logging: Logger = field(init=False, default=None)

    def __post_init__(self):
        if self.logging is None:
            self.logging = Logger(self.user)

Python: initiated Logger with dataclass field param

2 Answers2