0

I am looking desperately for a solution for the following problem: For a project, I need multiple independent loggers let's say one for logging readable progress for the user to console (stderr) and one for collecting some stats into a json output file. I am using structlog and my example below shows the problem

import structlog

logger1 = structlog.get_logger()
logger1.info('make something that needs to be logged', x=1, y=2, z='abc')

logger2 = structlog.get_logger()
structlog.configure(processors=[structlog.processors.JSONRenderer()])
logger2.info('structured_output', a=[1, 2, 3], b={'a': 1})

logger1.info('now this one is also json but it shouldn\'t')

gives the output

2022-06-08 08:30.34 [info     ] make something that needs to be logged x=1 y=2 z=abc
{"a": [1, 2, 3], "b": {"a": 1}, "event": "structured_output"}
{"event": "now this one is also json but it shouldn't"}

logger1 works fine until I apply the configuration for json rendering.

What I would like to see is

2022-06-08 08:30.34 [info     ] make something that needs to be logged x=1 y=2 z=abc
{"a": [1, 2, 3], "b": {"a": 1}, "event": "structured_output"}
2022-06-08 08:39.41 [info     ] still no json output, great!

How can I keep both loggers separate so that the configuration for logger1 is unchanged?

Thanks! Max

Max
  • 335
  • 1
  • 3
  • 12

2 Answers2

2

This is a bit more involved, but it's possible using structlog's most powerful standard library integration strategy, where every log entry goes through the standard library and that allows you to use the standard library for:

  1. multiplexing
  2. routing the log entries by logger name.

Multiplexing is demonstrated at the bottom of the linked above, with an example of how to have colorful logs in the terminal and plain ones into a file.

What's missing now is to dispatch between them based on the name of the logger. You'd do something like structlog.[stdlib.]get_logger("json-only") to set it to a well-known name that is used a logger name and then can be referred to in standard library logging config.

It's not as straight-forward as I'd like it to be, but the example in the docs should provide a good place to start.

hynek
  • 3,647
  • 1
  • 18
  • 26
  • first of all thanks for the quick response! I tried the example from the docs and I can have to separate loggers. However, what I don't manage so far is that the human readable console logger also supports events and that I only change the processor to JSONRenderer for the `json-only` logger... I appreciate any further help :-) – Max Jun 09 '22 at 07:20
  • I'm not 100% sure what you mean by "supports events"? Replacing `structlog.dev.ConsoleRenderer(colors=False)` by `structlog.processors.JSONRenderer()` should work? – hynek Jun 09 '22 at 14:29
0

nevermind, I was overcomplicating things... I leave the structlog logging to console as it is and write a small custom json-report class that does the trick. Thanks for your help though!

Max
  • 335
  • 1
  • 3
  • 12