Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
1
vote
1 answer

Spring boot Kafka request-reply scenario

I am implementing POC of request/reply scenario in order to move event-based microservice stack with using Kafka. There is 2 options in spring. I wonder which one is better to use. ReplyingKafkaTemplate or cloud-stream First is ReplyingKafkaTemplate…
fatih
  • 105
  • 9
1
vote
1 answer

Why does kafkaTemplate send methods not work with a nullable key?

https://docs.spring.io/spring-kafka/docs/current/api/org/springframework/kafka/core/KafkaTemplate.html#sendDefault(K,V) Calling above method from kotlin with a nullable key results in compiler warnings. Shouldn't the key be nullable? It is…
Jacob Botuck
  • 411
  • 6
  • 22
1
vote
0 answers

Confluent Kafka JSON Deserialization Error

I have a.NET application which is writing a JSON message to Kafka TOPIC using the Confluent JSON schema aware serializer. I have another spring boot JAVA application which is reading from that topic and creating a KTABLE and grouping by Keys and…
NARASIMHA MURTHY
  • 105
  • 1
  • 13
1
vote
0 answers

Spring Batch: One Reader, two processors and two kafkawriters

My scenario is +---> ItemProcessor#1 ---> KafkaItemWriter#1 | ItemReader ---> item ---+ | +---> ItemProcessor#2 ---> KafkaItemWriter#2 ItemReader is…
user4005166
  • 23
  • 4
1
vote
1 answer

How to find the processing time of Kafka messages?

I have an application running Kafka consumers and want to monitor the processing time of each message consumed from the topic. The application is a Spring boot application and exposes Kafka consumer metrics to Spring Actuator Prometheus endpoint…
Vinod Kumar Payakat Venugopal
  • 41
  • 6
1
vote
0 answers

Kafka consumer is getting out of active group and not able to consume ant message further

I am using spring kafka to implement kafka listener with default properties. I have noticed that with some failure eg: broker not available or any internal call failure after consuming message, it stops consuming messages and I can see attached…
jagmohansharma
  • 61
  • 1
  • 6
1
vote
2 answers

knowing that kafka has processed all the messages

How to know that Kafka has processed all the messages? is there any command or log file that species Kafka offset under processing and the last Kafka offset?
aymen0406
  • 127
  • 10
1
vote
1 answer

KafkaItemWriter writing but not to default topic

Can someone post an example or help how i can use KafkaItemWriter to write java objects into non-default topic. Just like in SpringKafka we use KafkaTemplate.send(topicName, user) is there a way for kafkaItemWriter also? I do not want to write to…
user4005166
  • 23
  • 4
1
vote
1 answer

Retry with backoff time per consumed message that fails in Kafka

In Kafka, is it possible to set a backoff time per message, if processing of that message fails ? - So that I can process other messages, and try again later with the message that failed ? If I just put it back out in front of the topic, it will…
user1511956
  • 784
  • 3
  • 9
  • 22
1
vote
1 answer

How to get Kafka brokers in a cluster using Spring Boot Kafka?

I have a Spring Boot (2.3.3) service using spring-kafka to currently access a dedicated Kafka/Zookeeper configuration. I have been using the application.properties setting spring.kafka.bootstrap-servers=localhost:9092 to access my dev/test Apache…
user3175414
  • 173
  • 1
  • 7
1
vote
1 answer

Spring-Kafka consumer's listener method is not called due to parse error

I'm trying to implement a Kafka consumer using Kafka-Spring (spring-boot). I send a message using a producer but Spring throws an exception and my listener method is not called because of this exception. Here's my listener method: @Service public…
xyzt
  • 1,201
  • 4
  • 18
  • 44
1
vote
1 answer

java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce at kafka.utils.TestUtils.tempDir(TestUtils.scala)

While using @EmbeddedKafka(topics = { "checkins" }) I am getting an error Caused by: java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce at kafka.utils.TestUtils.tempDir(TestUtils.scala) at…
Rohit Naik
  • 501
  • 1
  • 4
  • 14
1
vote
0 answers

Spring Boot Out of Memory Due to Kafka Common Metrics

We have a Spring Boot app, which went Out of Heap Memory Last Week. Since the HeapDumpOnOutOfMemory flag was enabled, we could see in a 4GB Heap, around 3.7GB space was occupied by com.sun.jmx.mbeanserver.NamedObject. All these Objects had key/value…
Apurv
  • 315
  • 5
  • 14
1
vote
1 answer

Topic sharing between consumers

I am working on a Springboot application where it rebalances for different partitions in Kafka topics (eg. 700 topics with 10 paritions each i.e. 7000 partitions). But I want to spin up multiple instances of dockerized app where the application will…
S. Arora
  • 11
  • 5
1
vote
1 answer

How to log offset in KStreams Bean using spring-kafka and kafka-streams

I have referred almost all the questions regarding logging offset on KStreams via Processor API's transform() or process() method like mentioned in many questions here - How can I get the offset value in KStream But Im not able to get the solution…
user2340345
  • 793
  • 4
  • 16
  • 38
1 2 3
…
99 100