Using python Faust, all of my logs are doubled.
As loggers are hierarchal, i want to prevent the downstream logger father.son from logging anything and propagate all the logs to upstream father logger.
I tried:
1 - setting disable_existing_loggers to True in the dictConfig when main is called
2 - adding filter (inside a handler) to a specific logger name using the dictConfig:
class LogPreventingFilter(logging.Filter):
def filter(self, record):
return False
logging_config = {
'version': 1,
'disable_existing_loggers': True,
'filters': {
'log_preventing_filter': {
'()': LogPreventingFilter
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'DEBUG',
'filters': ['log_preventing_filter']
}
},
'loggers': {
'father.son': {
'level': 'DEBUG',
'handlers': ['console']
}
}
}
dictConfig(logging_config)
3 - adding the same logging_config to faust's app configurations logging_config (with and without 'disable_existing_loggers': True) - in that case i could see logs from the logger i mentioned (father.son) but not from the father, even after i added the father logger to to faust config), nor logs from any other non-hierarichial logger ( like getLogger("some_name_not_related_to_father")).
ill mention that i tried to remove the filter and handler and it didnt make any change.
Worth to mention that the logs start to double only after i call faust's app.main().
also worth mentioning that the loggers that i want to actually log data are all from imported modules, not part of the code that is visible for faust. only father.son logger is.
my code:
some.imported.module
father_logger = logging.getLogger("father")
some_other_logger = logging.getLogger("not_hierarchiali_related")
class MyGenericClass:
// add filters and handlers and do some generic stuff
stream_processing_module
son_logger = logging.getLogger("father.son")
async def processing_function():
// do some stuff
son_logger.info("log this and that")
the module using faust:
son_logger = logging.getLogger("father.son")
@app.agent
async def stream_processing():
// call the processing_function() for the other module for each record
son_logger.info("log some data")
if __name__ == "main":
son_logger.info("start")
with MyGenericClass:
app.main()
notice that only son_logger is visible to faust.
still - i want the father logger to do the printing and not the son logger.
any ideas? thanks!
