Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
1
vote
1 answer

What is the difference between producing message to kafka topic with Spring Integration MessageChannel and KafkaProducer?

I see there are two approaches to produce message to kafka topic, one with KafkaProducer.send(ProducerRecord record) and other with(springframework.message) MessageChannel.send(Message). Former returns Future and latter returns boolean. I want…
Priya
  • 143
  • 4
  • 17
1
vote
1 answer

metrics method of MessageListenerContainer is not capturing the right values

I am using spring-kafka 2.2.8 to created a batch consumer and trying to capture the my container metrics to understand the performance details of the batch consumer. @Bean public ConsumerFactory consumerFactory(){ return new…
Raj
  • 1,467
  • 3
  • 23
  • 51
1
vote
1 answer

Spring Boot: how to idiomatically configure Schema Registry Serdes in spring-kafka

Are there examples of configuring SpecificAvroSerdes (or any schema registry-based serdes - JsonSchema and Protobuf) in spring-kafka that allow leveraging some of the autoconfiguration (based on yaml or properties files). There are a few similar…
Vik Gamov
  • 5,446
  • 1
  • 26
  • 46
1
vote
2 answers

Spring Cloud Streams kafka binder - topic serialization configuration

So I think I've run myself into confusion as I understand there are two different kafka binders for SpringCloudStreams: Spring Cloud Streams Kafka Binder Spring Cloud Streams Kafka Streams Binder I'm looking for the correct YAML settings to define…
Jeef
  • 26,861
  • 21
  • 78
  • 156
1
vote
1 answer

Apache Kafka - REST API based consumer?

I followed this article to build a simple Java Spring Boot application to work with Apache Kafka. Defined the Producer controller as mentioned below @RestController @RequestMapping(value = "/kafka") public class KafkaController { private final…
One Developer
  • 99
  • 5
  • 43
  • 103
1
vote
1 answer

Exception while testing Spring Kafka error handler for @KafkaListener

I am trying to test my error handler defined in the containerlistenerfactory which is being used by @KafkaListener. I have different retry counts based of different exception throw by the listener which i want to test is working correctly. But i am…
user1945064
  • 199
  • 3
  • 4
  • 18
1
vote
1 answer

How to read a message from Kafka topic on demand

How can I read a message from Kafka topic on demand. I have the topic name, offsetId, PartitionID, using these three params, how can i retrieve a specific message from Kafka Topic. Is it possible using Spring Kafka ? I am using spring boot…
AJK1305
  • 109
  • 2
  • 15
1
vote
0 answers

Will Kafka Producer always waits for the value specified by linger.ms, before sending a request?

As per LINGER_MS_DOC in ProducerConfig java class: "The producer groups together any records that arrive in between request transmissions into a single batched request. Normally this occurs only under load when records arrive faster than they can…
hsingh
  • 661
  • 5
  • 26
1
vote
2 answers

Spring 5.X web Kafka Integeration using xml configuration (Not Spring boot)

I am trying to setup a kafka consumer on legacy web application using xml configuration that is running Spring 5.2 release project (It is not a spring boot project). After looking up I found a project that sets up the kafka consumer using xml…
Praveen
  • 557
  • 1
  • 5
  • 20
1
vote
1 answer

How to set change the retry attempts based of the exception type in Kafka SeekToCurrentErrorHandler?

I have a spring boot application and i am trying to create a Kafka retry listener wherein, based of the exception type i need to change the retry attempts. For example: If exception type is A then the retry attempts should be 5 If exception type is…
user1945064
  • 199
  • 3
  • 4
  • 18
1
vote
1 answer

Consumer application causing duplicate while reading records from multiple partition

I am trying to consume record from stream using spring-kafka. The records are in the avro schema. As there are 2 partitions on the topic, I am using kafka concurrency as 2 to consume records from partitions in parallel. But, it seems it is causing…
Praveenks
  • 1,436
  • 9
  • 40
  • 79
1
vote
0 answers

Kafka topic consumer service - listener method threw an exception java.lang.ArrayIndexOutOfBoundsException

I have a Spring based service which subscribes Kafka topic. Service works fine but sometime service stops working with the following error. In such error scenarios, service stop subscribing topic and throws below error. If I give new consumer…
Brijesh
  • 11
  • 2
1
vote
2 answers

Spring boot Kafka polling until queue is empty and sleep

I am using spring boot with apache kafka. I have a dead letter queue that I want to poll from partition y every x hours, poll until there are no messages left and go back to sleep until the next interval. Is there an option to do that?
jedam
  • 11
  • 2
1
vote
1 answer

MessageConversionException during serialization in spring-kafka?

To show what I mean I make simple project. Dependencies: org.springframework.boot spring-boot-starter-web …
faoxis
  • 1,912
  • 5
  • 16
  • 31
1
vote
1 answer

Get response from Kafka producer after each send command

I am using spring boot Kafka to send messages to a topic. My requirement is to read data incrementally from a table and publish it into a topic based on a date time field and as it is a scheduled process i need to store the date-time field from each…
Gaurav singh
  • 61
  • 6
1 2 3
…
99 100