Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
13
votes
1 answer

Exponential backoff with message order guarantee using spring-kafka

I'm trying to implement a Spring Boot-based Kafka consumer that has some very strong message delivery guarentees, even in a case of an error. messages from a partition must be processed in order, if message processing fails, the consumption of the…
Maciej Papież
  • 431
  • 1
  • 6
  • 20
13
votes
4 answers

Transaction Synchronization in Spring Kafka

I want to synchronize a kafka transaction with a repository transaction: @Transactional public void syncTransaction(){ myRepository.save(someObject) kafkaTemplate.send(someEvent) } Since the merge…
Eike Behrends
  • 855
  • 2
  • 7
  • 11
13
votes
1 answer

Spring Kafka Consumer Retry

I am using Spring Kafka consumer which fetches messages from a topic and persist them into a db. If a failure condition is met , say for instance the db is unavailable , does kafka consumer library provide mechanism to retry ? If it does , is there…
Punter Vicky
  • 15,954
  • 56
  • 188
  • 315
13
votes
1 answer

Enabling @KafkaListener to take in variable topic names from application.yml file

I am attempting to load in multiple topics to a single @KafkaListener but am running into trouble as I believe it is looking for a constant value, but initializing the topics variable from the application.yml file causing something issues, I was…
terrabl
  • 743
  • 3
  • 8
  • 23
13
votes
3 answers

Spring Kafka: Multiple Listeners for different objects within an ApplicationContext

Can I please check with the community what is the best way to listen to multiple topics, with each topic containing a message of a different class? I've been playing around with Spring Kafka for the past couple of days. My thought process so…
yfl
  • 347
  • 2
  • 3
  • 10
12
votes
1 answer

Kafka consumer reconnection after getting disconnected

I have my project set up using Spring Boot and Spring Kafka, and there are three consumers. Checking the logs, I can see that from time to time the consumers get disconnected: catalina.out:2019-04-27 02:19:57.962 INFO 18245 --- [ntainer#2-0-C-1]…
Hua
  • 666
  • 2
  • 9
  • 21
12
votes
4 answers

How can I retry failure messages from kafka?

My spring-boot application(consumer) process messages from Apache Kafka. Periodically, massage can't process and consumer throw exception. Consumer commits offset anyway. Can I distinguish success messages from failure messages in Kafka? I think,…
ip696
  • 6,574
  • 12
  • 65
  • 128
12
votes
1 answer

Spring Boot Kafka Listener vs Consumer

What's the difference? Can the term KafkaConsumer and KafkaListener be used interchangeably?
George
  • 2,820
  • 4
  • 29
  • 56
12
votes
3 answers

How to pass dynamic topic name to @KafkaListener(topics) from environment variable

I'm writing a Kafka consumer. I need to pass the environment variable topic name to @KafkaListener(topics = ...). This is what I have tried so far: import org.springframework.beans.factory.annotation.Autowired; import…
CodeCool
  • 193
  • 2
  • 12
12
votes
3 answers

springboot-kafka java 8 time serialization

Currently working with spring-boot 2.0.4 with spring-kafka 2.1.8.RELEASE. I've wanted to simplify the interchange a bit sending objects to kafka template and used json as format. Some of the messages that needs to be deserialized however contains…
geneqew
  • 2,401
  • 5
  • 33
  • 48
12
votes
3 answers

The class is not in the trusted packages although appears in the list of trusted packages

I am trying to implement a simple Kafka communication between 2 different Spring Boot applications with out any special settings, this application has only one kafkalistener. My yml for the consumer is the following: spring: kafka: …
HeyItsMe
  • 191
  • 1
  • 1
  • 10
12
votes
4 answers

How to expose kafka metrics to /actuator/metrics with spring boot 2

I was looking a while and didn't seem to find the answer. I'm using Spring boot 2, Spring Kafka 2.1.4 and I want to see the kafka consumer metrics in the /metrics endpoint of spring boot actuator. What I don't understand is - should I implenemt the…
Michael Azimov
  • 131
  • 1
  • 1
  • 4
12
votes
5 answers

Spring Kafka Producer not sending to Kafka 1.0.0 (Magic v1 does not support record headers)

I am using this docker-compose setup for setting up Kafka locally: https://github.com/wurstmeister/kafka-docker/ docker-compose up works fine, creating topics via shell works fine. Now I try to connect to Kafka via spring-kafka:2.1.0.RELEASE When…
DerM
  • 1,497
  • 1
  • 14
  • 27
12
votes
1 answer

Acknowledgement.acknowledge() throwing exception in spring-kafka @KafkaListener

When I set enable.auto.commit to false and try to manually commit offset using annotation based spring-kafka @KafkaListener, I get a org.springframework.kafka.listener.ListenerExecutionFailedException: Listener method could not be invoked with the…
nYmERioN805
  • 125
  • 1
  • 1
  • 7
11
votes
0 answers

Why do I get this log - Resetting the last seen epoch

I have a spring boot application that sends multiple topics to kafka. In the service I use to send I have autowired this one. private final KafkaTemplate kafkaTemplate; WHen I use this kafkaTemplate to send to multiple topics I…
Ole Bille
  • 469
  • 2
  • 14
1 2 3
…
99 100