0

While creating the KafkaAdminClient

    client = KafkaAdminClient(bootstrap_servers=bootstrap_servers,
                          security_protocol=security_protocol,
                          sasl_mechanism=SASL_MECHANISM,
                          sasl_plain_username=username,
                          sasl_plain_password=password,
                          )

there are a lot of debug and info logs that are generated like:

 kafka.client [DEBUG]- [client_async.py:279 -   _conn_state_change()] - Node bootstrap-0 connected
 kafka.protocol.parser [DEBUG]- [parser.py:59 -         send_request()] - Sending request MetadataRequest_v0(topics=[])

Is there a config that would allow switching off DEBUG and INFO logs and only show WARN or ERROR logs?

Tushar
  • 528
  • 4
  • 20

2 Answers2

3

It uses the standard logging library and the client side handler is named kafka:

import logging
logger = logging.getLogger('kafka')
logger.setLevel(logging.WARN)

That should configure the kafka-python library log level.

Gerard Garcia
  • 1,554
  • 4
  • 7
0

If you want to silence certain kafka logs over others you can fetch loggers for the specific sources listed next to your log levels. For example, this helped when my consumer was flooding our system with fetcher.py logs, but I wanted to avoid impacting the logs for other kafka uses.

import logging
logging.getLogger('kafka.consumer.fetcher').setLevel(logging.INFO)
quinn maloney
  • 11
  • 4