Python - Logging from multiple modules using structlog

Question

I am trying to use Structlog to to log to a file and then using filebeat to send the log to my logging service.

I have made the everything work, but I would like to be able to use the same logger across multiple modules, like with Pythons default logger (see https://docs.python.org/2/howto/logging.html section "Logging from multiple modules").

One of the reasons is that I would like to bind a sessionID to my logoutput that should be logged across all modules called from this session.

Probably need some fundamental knowledge on how to use structlogger, but havn't found the answer in their documentation or in other posts.

Please advice....

An example:

main.py

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

import uuid

from mylogger import myLogger
import otherModule

myLogger = myLogger()

myLogger.log.warning('working', error='test')

myLogger.log = myLogger.log.bind(sessionID=str(uuid.uuid4()))

myLogger.log.warning('Event where sessionID is bound to logger', error='test')

otherModule = otherModule.otherModule()

myLogger.py

#!/usr/bin/python3
# -*- coding: UTF-8 -*-

import datetime, logging
from structlog import wrap_logger, get_logger
from structlog.processors import JSONRenderer
from structlog.stdlib import filter_by_level, add_log_level


class myLogger():
    def __init__(self, loggername="root", logLevel='INFO', logFile='test2.log'):

        self.log = wrap_logger(
            logging.getLogger('root'),
            processors=[
                filter_by_level,
                add_log_level,
                self.add_timestamp,
                JSONRenderer(indent=1, sort_keys=True)
            ]
        )

        logging.basicConfig(format='%(message)s', level=logLevel, filename=logFile)

    def my_get_logger(self, loggername="AnyLogger"):
        log = get_logger(loggername)

        return log

    def add_timestamp(self,  _, __, event_dict):
        event_dict['timestamp'] = datetime.datetime.utcnow().isoformat()
        return event_dict

otherModule.py

import structlog
from mylogger import myLogger

class otherModule():
    def __init__(self):
        logger = structlog.get_logger('root')
        ## This logger does not have the processors nor the bound sessionID

        logger.warning('In other module')
        ## This logmessage is printed to console

        logger2 = myLogger();
        ## This logger has all the processors  but not the bund sessionID

        logger2.log.warning('In other module')
        ## This log is written to my logfile, but without the sessionID

score 2 · Answer 1 · answered Sep 26 '18 at 14:59

You need to use a wrapped dictionary as a context class as explained in the structlog documentation

So you will end up with something like this:

structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        # other processors
    ],
    context_class=structlog.threadlocal.wrap_dict(dict),
)

Python - Logging from multiple modules using structlog

1 Answers1