Python logging: propagate messages of level below current logger level

Question

I want to log messages of a specific logger name, of a certain level and higher (say INFO and up) to a specific log handler, say a file handler, while still getting all log messages to the console. Python is version 2.7.

What I tried until now was to create two loggers:

A root logger
A named logger

For the root logger, I attached a logging.StreamHandler, and set the log level to logging.DEBUG.

Then I attached a handler to the named logger and set level to logging.INFO for that logger.

When I now call my module, which uses the named logger, I do not get DEBUG logs propagated to the root logger any more.

Note: the extraLogger has a StreamHandler here to demonstrate the issue. In my production code I'd use a FileHandler

import logging

def do_logging(turn):
    logger = logging.getLogger('extra')
    logger.info('some info turn %d' % turn) 
    logger.debug('this is debug fudge turn %d' % turn)

rootLogger = logging.getLogger()
handler = logging.StreamHandler()
rootFormatter = logging.Formatter('root - %(levelname)s: %(msg)s')
handler.setFormatter(rootFormatter)
rootLogger.addHandler(handler)
rootLogger.setLevel(logging.DEBUG)

do_logging(1)

extraLogger = logging.getLogger('extra')
extraHandler = logging.StreamHandler()
extraFormatter = logging.Formatter('extra - %(levelname)s: %(msg)s')
extraHandler.setFormatter(extraFormatter)
extraLogger.addHandler(extraHandler)
extraLogger.setLevel(logging.INFO)

do_logging(2)

Actual Output:

root - INFO: some info turn 1
root - DEBUG: this is debug fudge turn 1
extra - INFO: some info turn 2
root - INFO: some info turn 2

Output that I would like to have:

root - INFO: some info turn 1
root - DEBUG: this is debug fudge turn 1
extra - INFO: some info turn 2
root - INFO: some info turn 2
root - DEBUG: this is debug fudge turn 2

I suspect that a custom Filter would be helpful in this case, but I do not know how...

score 6 · Accepted Answer · edited May 23 '17 at 10:27

6

You could use robert's LevelFilter like this:

# Put the Filter on the Handler so only INFO and higher is handled
extraHandler.addFilter(LevelFilter(logging.INFO))

# Let the Logger process everything (so it can propagate records to root)
extraLogger.setLevel(logging.DEBUG)

import logging

class LevelFilter(logging.Filter):
    """
    https://stackoverflow.com/a/7447596/190597 (robert)
    """
    def __init__(self, level):
        self.level = level

    def filter(self, record):
        return record.levelno >= self.level

def do_logging(turn):
    logger = logging.getLogger('extra')
    logger.info('some info turn %d' % turn) 
    logger.debug('this is debug fudge turn %d' % turn)

rootLogger = logging.getLogger()
handler = logging.StreamHandler()
rootFormatter = logging.Formatter('root - %(levelname)s: %(msg)s')
handler.setFormatter(rootFormatter)
rootLogger.addHandler(handler)
rootLogger.setLevel(logging.DEBUG)
do_logging(1)

extraLogger = logging.getLogger('extra')
extraHandler = logging.StreamHandler()
extraFormatter = logging.Formatter('extra - %(levelname)s: %(msg)s')
extraHandler.setFormatter(extraFormatter)
extraLogger.addHandler(extraHandler)

# Put the Filter on the Handler so only INFO and higher is handled
extraHandler.addFilter(LevelFilter(logging.INFO))

# Handle everything (so it can propagate to root)
extraLogger.setLevel(logging.DEBUG)
do_logging(2)

edited May 23 '17 at 10:27

Community

1
1

answered Aug 05 '13 at 13:32

unutbu

842,883
184
1,785
1,677

1

Works fine, thank you very much. Seems like after a logger discards log entries based ont the bare level mechanism, it does not propagate them any more. Here this is worked around by setting the log level lower, but only log messages of the desired level, which lets log entries of lower level propagate. – trapicki Aug 05 '13 at 14:42
1

Yes, that's correct. There is a [nice flow chart in the docs](http://docs.python.org/2/howto/logging.html#logging-flow) which sets the right mental model. You don't want the filter attached to the logger to reject the record because then it never gets to the propagate step. – unutbu Aug 05 '13 at 17:53
1

Oh, there's a big difference if a filter is attached to a handler or a logger... The _handler sub-chart_ seems misleading: _Stop_ would not stop processing, just return from that handler without emitting something, and the flow would pass on the the next handler, or am I wrong, @unutbu? – trapicki Aug 06 '13 at 09:32
You are right; `Stop` does not stop processing. It just stops that sub-chart. And there is a loop involved, as there could be multiple handlers. So flow of control goes to the next handler if there is one, and finally back to the main flow chart. For the precise story [go to the source](http://hg.python.org/cpython/file/80e9cb6163b4/Lib/logging/__init__.py#l1434). – unutbu Aug 06 '13 at 09:59

score 5 · Answer 2 · edited Mar 20 '22 at 20:51

propagate

If this attribute evaluates to true, events logged to this logger will be passed to the handlers of higher level (ancestor) loggers, in addition to any handlers attached to this logger. Messages are passed directly to the ancestor loggers’ handlers - neither the level nor filters of the ancestor loggers in question are considered.

If this evaluates to false, logging messages are not passed to the handlers of ancestor loggers.

please visit official python site for detailed discussion regarding this.

Diabling the propagate message

import logging

handler = logging.StreamHandler()

parent = logging.getLogger("parent")
parent.addHandler(handler)
child = logging.getLogger("parent.child")
child.propagate = False

child.setLevel(logging.DEBUG)
child.addHandler(handler)

child.info("HELLO")

Output:

$ python3.10 propagate.py 
HELLO

Code without disabling the propagate message

import logging

handler = logging.StreamHandler()

parent = logging.getLogger("parent")
parent.addHandler(handler)
child = logging.getLogger("parent.child")
#child.propagate = False

child.setLevel(logging.DEBUG)
child.addHandler(handler)


child.info("HELLO")

Output:

$ python3.10 propagate.py 
HELLO
HELLO

Note: Be aware that also the [disabled](https://docs.python.org/3/library/logging.config.html?highlight=disable_existing_loggers) attribute of a logger can hinder the propagation. A logger might be disabled without you explicitly setting it. Took me some time. — Code_beginner, Mar 09 '23 at 09:56

score 2 · Answer 3 · answered Aug 05 '13 at 13:28

2

There is a method in the logger class called "propagate" that seems to do what you ask: http://docs.python.org/2/library/logging.html#logger-objects

answered Aug 05 '13 at 13:28

schme

21
1

1

AFAIU `Logger.propagate` is a boolean property that controls if propagation takes place at all. I've tried to fiddle with that, but it seems log entries are discarded based on the level and thus do not propagate to the root logger anyway. – trapicki Aug 05 '13 at 14:45

Python logging: propagate messages of level below current logger level

3 Answers3

propagate

Diabling the propagate message

Output:

Code without disabling the propagate message

Output:

Linked