Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
15
votes
3 answers

The correct way for creation of KafkaTemplate in spring boot

I try configure apache kafka in spring boot application. I read this documentation and follow the steps: 1) I add this lines to aplication.yaml: spring: kafka: bootstrap-servers: kafka_host:9092 producer: key-serializer:…
ip696
  • 6,574
  • 12
  • 65
  • 128
15
votes
1 answer

Write to two Kafka topics in a single transaction using Spring Kafka

I'm trying to work out if there's a way of using Kafka's transaction feature to write to two topics within a transaction. I know the typical scenario to use Kafka's transactions is in a consumer-producer pattern and that seems well documented. What…
SteveD
  • 5,396
  • 24
  • 33
15
votes
2 answers

Swagger documentation for Kafka Listener

I have used Swagger to generate a document for rest API, but I am building Kafka listener and wanted to generate a document for it. Do we have any possibility using Swagger or something similar? Thanks, Manish
krmanish007
  • 6,749
  • 16
  • 58
  • 100
15
votes
2 answers

java.lang.NoSuchMethodError on Kafka Consumer with spring-kafka 2.1.0 and SpringBoot 1.5.9

I am trying to setup Kafka Consumer using SpringBoot(1.5.9) and Spring-kafka(2.1.0). However when I start my app I get java.lang.NoSuchMethodError: org.springframework.util.Assert.state(ZLjava/util/function/Supplier;)V on Kafka…
KafkaConsumer
  • 183
  • 1
  • 1
  • 8
14
votes
1 answer

How to mock result from KafkaTemplate

I have a method for sending kafka message like this: @Async public void sendMessage(String topicName, Message message) { ListenableFuture> future = kafkaTemplate.send(topicName, message); future.addCallback(new…
Bali
  • 705
  • 4
  • 13
  • 21
14
votes
2 answers

Spring Kafka JsonDesirialization MessageConversionException failed to resolve class name Class not found

I have two services that should communicate via Kafka. Let's call the first service WriteService and the second service QueryService. On the WriteService side, I have the following configuration for producers. @Configuration public class…
CROSP
  • 4,499
  • 4
  • 38
  • 89
14
votes
3 answers

Kafka: The message when serialized is larger than the maximum request size you have configured with the max.request.size configuration

Getting the following error (Kafka 2.1.0): 2018-12-03 21:22:37.873 ERROR 37645 --- [nio-8080-exec-1] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='null' and payload='{82, 73, 70, 70, 36, 96, 19,…
sapy
  • 8,952
  • 7
  • 49
  • 60
14
votes
5 answers

Is there a code sample for multiple producers in spring kafka?

I have an application that may need multiple producers. All code samples I see seem to support a single producer, reading config from app during app startup. If there are multiple producers and we want to pass in different producer config, is there…
Merrin
  • 655
  • 2
  • 7
  • 18
14
votes
3 answers

Problems adding multiple KafkaListenerContainerFactories

Hi I'm currently dabbling in Spring Kafka and succeeded in adding a single KafkaListenerContainerFactory to my listener. Now I'd like to add multiple KafkaListenerContainerFactorys (One for a topic that will have messages in json, another one for…
Omar Andres Olivares Rodriguez
  • 97
  • 2
  • 6
  • 12
13
votes
7 answers

EmbeddedKafka failing since Spring Boot 2.6.X : AccessDeniedException: ..\AppData\Local\Temp\spring.kafka*

e: this has been fixed through Spring Boot 2.6.5 (see https://github.com/spring-projects/spring-boot/issues/30243) Since upgrading to Spring Boot 2.6.X (in my case: 2.6.1), I have multiple projects that now have failing unit-tests on Windows that…
DRoppelt
  • 406
  • 1
  • 4
  • 14
13
votes
3 answers

List Kafka Topics via Spring-Kafka

We would like to list all Kafka topics via spring-kafka to get results similar to the kafka command: bin/kafka-topics.sh --list --zookeeper localhost:2181 When running the getTopics() method in the service below, we get…
Paul Croarkin
  • 14,496
  • 14
  • 79
  • 118
13
votes
1 answer

How to handle SerializationException after deserialization

I am using Avro and Schema registry with my Spring Kafka setup. I would like to somehow handle the SerializationException, which might be thrown during deserialization. I found the following two…
kkflf
  • 2,435
  • 3
  • 26
  • 42
13
votes
1 answer

Difference between group id, Client id and id in KafkaListener Spring Boot

I am starting to work with Spring Boot 2 and Spring Kafka, I don't quite understand what's the difference between group id, Client id, and id in in KafkaListener interface. I know group ID is used by Kafka broker to manage multiple Consumer in the…
Am1rr3zA
  • 7,115
  • 18
  • 83
  • 125
13
votes
7 answers

How to use Micrometer Timer to record duration of async method (returns Mono or Flux)

I'd like to use Micrometer to record the execution time of an async method when it eventually happens. Is there a recommended way to do this? Example: Kafka Replying Template. I want to record the time it takes to actually execute the…
Mark
  • 4,970
  • 5
  • 42
  • 66
13
votes
3 answers

kafka embedded : java.io.FileNotFoundException: /tmp/kafka-7785736914220873149/replication-offset-checkpoint.tmp

I use kafkaEmbedded in integration test and I get FileNotFoundException : java.io.FileNotFoundException: /tmp/kafka-7785736914220873149/replication-offset-checkpoint.tmp at java.io.FileOutputStream.open0(Native Method) ~[na:1.8.0_141] at…
qasmi
  • 215
  • 5
  • 11
1 2
3
…
99 100