How to configure logging for Kafka producers?

Question

I am using Kafka producer client and i don't have any log4j configuration in my project.

On running, the program prints a lot of Kafka Debug logs which i really don't want.

So, i tried to add a log4j.properties to set log level to ERROR as below which does not seem to work:

log4j.rootLogger=ERROR

How do i change Kafka Log Level?

score 15 · Answer 1 · answered Apr 26 '17 at 23:30

Use the command line flag -Dlog4j.configuration=file:/path/to/log4j.properties when running your client.

Example log4j property files:

https://github.com/apache/kafka/blob/trunk/config/tools-log4j.properties
- Used by tools like mirror maker, really just stdout
https://github.com/apache/kafka/blob/trunk/config/log4j.properties
- Used by tools like kafka server

For mirror maker and other tools that result in a call to kafka-run-class.sh, you can use the env variable KAFKA_LOG4J_OPTS (set to something like -Dlog4j.configuration=file:/path/to/log4j.properties) to change the logging configuration. See: https://github.com/apache/kafka/blob/0.10.2/bin/kafka-run-class.sh#L158

Example of my log4j.properties file for mirror maker that I use for testing.

# https://github.com/apache/kafka/blob/trunk/config/tools-log4j.properties

log4j.rootLogger=DEBUG, stderr

log4j.appender.stderr=org.apache.log4j.ConsoleAppender
log4j.appender.stderr.layout=org.apache.log4j.PatternLayout
log4j.appender.stderr.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.stderr.Target=System.err

score 3 · Answer 2 · answered Nov 22 '16 at 23:38

3

Try adding logging.level.org.apache.kafka: DEBUG into your clients configuration properties. I am using Springboot and this is the format. Use appropriate format for your clients program.

answered Nov 22 '16 at 23:38

sivakadi

181
1
4

`Cannot set level: WARNING for 'org.apache.kafka'` any ideas? – forresthopkinsa Jan 22 '18 at 21:21

score 0 · Answer 3 · answered Mar 03 '16 at 16:11

0

I assumed you were talking about Kafka server logs. You can change log level to ERROR using following config

log4j.logger.kafka=ERROR, kafkaAppender

Hope this helps!

answered Mar 03 '16 at 16:11

avr

4,835
1
19
30

7

I am talking bout the logs that come up in my client program..not the kafka server as such... – nikel Mar 03 '16 at 17:35

score 0 · Answer 4 · answered Mar 28 '18 at 10:30

0

org.apache.log4j.Logger.getLogger("org").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("akka").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("kafka").setLevel(Level.WARN);

answered Mar 28 '18 at 10:30

tjeubaoit

986
7
7

score 0 · Answer 5 · answered Jun 19 '20 at 15:53

0

Define the logging level as below in application.yml file or your properties file.

logging:
  level:
    root: INFO
    org:
      apache:
        kafka: WARN

By defining the above, Kafka will print out only the warnings logs.

answered Jun 19 '20 at 15:53

Jestino Sam

526
2
12
31

score 0 · Answer 6 · answered Feb 25 '23 at 03:21

Spring allows us to configure the package-level logging as follows.

Steps

Identify your logs - (I don't need to get info logs of confluent kafka packages.)
Find out the package first - io.confluent.kafka
Set it in the application properties

logging.level.io.confluent.kafka=WARN

In your case, you need to set the below in your application properties. I added confluent kafka as well, if you use it.

logging.level.org.apache.kafka=WARN
logging.level.io.confluent.kafka=WARN

How to configure logging for Kafka producers?

6 Answers6

Linked