Logback Kafka Appender Not Writing Logs to Topic

Question

I am trying to do a simple test of Logback Kafka Appender. Following is what I have in my logback.xml which I have placed in src/main/resources:

<?xml version="1.0" encoding="UTF-8"?>

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> 
    <encoder> <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern> 
    </encoder> 
</appender>

<!-- This is the kafkaAppender -->
<appender name="kafkaAppender"
    class="com.github.danielwegener.logback.kafka.KafkaAppender">
    <encoder>
        <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
        </pattern>
    </encoder>
    <topic>test</topic>
    <keyingStrategy
        class="com.github.danielwegener.logback.kafka.keying.NoKeyKeyingStrategy" />
    <deliveryStrategy
        class="com.github.danielwegener.logback.kafka.delivery.AsynchronousDeliveryStrategy" />

    <!-- Optional parameter to use a fixed partition -->
    <!-- <partition>0</partition> -->

    <!-- Optional parameter to include log timestamps into the kafka message -->
    <!-- <appendTimestamp>true</appendTimestamp> -->

    <!-- each <producerConfig> translates to regular kafka-client config (format: 
        key=value) -->
    <!-- producer configs are documented here: https://kafka.apache.org/documentation.html#newproducerconfigs -->
    <!-- bootstrap.servers is the only mandatory producerConfig -->
    <producerConfig>bootstrap.servers=localhost:9092</producerConfig>

    <!-- this is the fallback appender if kafka is not available. -->
    <!-- <appender-ref ref="STDOUT" /> -->
</appender>

<logger name="com.my.package" level="info" additivity="false">
    <appender-ref ref="kafkaAppender" />
</logger>
<root level="info">
    <appender-ref ref="kafkaAppender" />
</root>

I have started a Kafka server on Windows with a separate Zookeeper. I have created a test topic as well.

Started Zookeeper on Windows with : zkServer.cmd

Started Kafka Server on Windows with : kafka-server-start.bat config\server.properties

Created a topic : kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Started a consumer on Windows with : kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test --from-beginning

In my Java class all I have is a logger inside a method.

private static final Logger logger = 
LoggerFactory.getLogger(FormController.class);
    logger.debug("DEBUG :: Inside do post method");
    logger.info("INFO :: Inside do post method");

My imports are:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

My pom.xml is as follows:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.my.package</groupId>
<artifactId>SampleIGApp</artifactId>
<packaging>war</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>SampleIGApp Maven Webapp</name>
<url>http://maven.apache.org</url>
<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>3.8.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>javax.servlet</groupId>
        <artifactId>javax.servlet-api</artifactId>
        <version>3.1.0</version>
        <scope>provided</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/log4j/log4j -->
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.17</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.7.21</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-simple</artifactId>
        <version>1.7.21</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients -->
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>1.0.1</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.github.danielwegener/logback-kafka-appender -->
    <dependency>
        <groupId>com.github.danielwegener</groupId>
        <artifactId>logback-kafka-appender</artifactId>
        <version>0.2.0-RC1</version>
    </dependency>
</dependencies>
<build>
    <finalName>MyApp</finalName>
</build>

Can someone please tell me as to what am I doing wrong? Even if I comment out STDOUT appender-name, then also logs keep getting written to console, which in my case is Tomcat's catalina.out. I would appreciate any help. Thank you.

Hello, did you manage to fix this issue? I'm facing the same. Spring Kafka works well, but not the logback project, I dont see anything in my console consumer — breakline, Dec 03 '21 at 12:21
Refer this. https://stackoverflow.com/questions/74594536/helidon-mp-how-to-send-slf4j-log-messages-to-kafka-broker — Suraj s Kalloli, Jul 06 '23 at 10:33

score 1 · Answer 1 · answered Jun 19 '18 at 14:35

The documentation says you need to add logback-classic as dependency:

<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.2.3</version>
    <scope>runtime</scope>
</dependency>

logback-kafka-appender

Logback Kafka Appender Not Writing Logs to Topic

1 Answers1