Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
1
vote
1 answer

How to include Type MetaData for Deserialization in Spring Kafka

I am doing deserialization at the Listener in Spring Kafka. But this assumes that the type information was included or sent by a Spring Kafka producer. In my case the Json is being sent across by the Debezium MySQLConnector and it does not add this…
Roger Alkins
  • 125
  • 11
1
vote
1 answer

kafka consumer distribute messages to consumers from a single partition | Alternative

I have four instances of the application running under a cluster, it means I have four consumers under one consumer group. There would be one single topic with 100's of partition. Now I need to read unique messages from a specific partition using…
user3373917
  • 55
  • 6
1
vote
1 answer

Spring Kafka Configuration for 2 different kafka cluster setups

In one of our spring-boot based services, we intended to connect to 2 different kafka clusters simultaneously. These clusters each have their own set of bootstrap-servers, topic configurations etc. They are nowhere related to each other as was the…
Mukund Jalan
  • 1,145
  • 20
  • 39
1
vote
1 answer

Spring Kafka Requirements for Supporting Multiple Consumers

As one would expect its common to want to have different Consumers deserializing in different ways off topics in Kafka. There is a known problem with spring boot autoconfig. It seems that as soon as other factories are defined Spring Kafka or the…
Roger Alkins
  • 125
  • 11
1
vote
1 answer

class java.lang.String cannot be cast to class java.util.List (java.lang.String and java.util.List are in module java.base of loader 'bootstrap')

Trying to use ReplyingKafkaTemplate to return the response from the service, but keep getting an error java.lang.ClassCastException: class java.lang.String cannot be cast to class java.util.List (java.lang.String and java.util.List are in module…
San Jaisy
  • 15,327
  • 34
  • 171
  • 290
1
vote
1 answer

Failed to start bean 'replyingTemplate'; nested exception is java.lang.IllegalStateException:Error handler is not compatible with the message listener

I am trying to use ReplyingKafkaTemplate from the this example https://docs.spring.io/spring-kafka/docs/2.5.3.RELEASE/reference/html/#replying-template public List GetProducts() { ProducerRecord record = new…
San Jaisy
  • 15,327
  • 34
  • 171
  • 290
1
vote
2 answers

How to programmatically get topics permissions [Kafka, Java]

I'm implementing advanced Kafka health-check. Now it's realized "standard" health-check: @Override protected void doHealthCheck(Builder builder) { try (AdminClient adminClient = AdminClient.create(this.kafkaAdmin.getConfig())) { …
Torino
  • 445
  • 5
  • 12
1
vote
1 answer

Spring Kafka MessageListenerContainer Resume/Pause # spring-kafka

As native KafkaConsumer is not thread safe, so it is discouraged to call pause and resume methods from different thread instead of kafka-consumer processing thread. but as spring-kafka provides another layer KafkaMessageListenerContainer which…
SBhogal
  • 119
  • 2
  • 15
1
vote
2 answers

Spring Boot - Kafka Consumer Bean Scope

I'm using a CacheManager in a Spring Boot application with SCOPE_REQUEST scope. @Bean @Scope(value = WebApplicationContext.SCOPE_REQUEST, proxyMode = ScopedProxyMode.TARGET_CLASS) public CacheManager cacheManager() { return new…
Arnold Robert Turdean
  • 207
  • 3
  • 15
1
vote
2 answers

Send / Receive Java Objects through Kafka

I want to send Java Object through Kafka and receive them as Java Object as well. I thought the following configuration for ProducerFactory and ConsumerFactory will suffice, but I am receiving payload as…
definepi314
  • 63
  • 1
  • 10
1
vote
1 answer

How to use ABSwitchCluster to failover between Kafka clusters

I have an application that is using Kafka to Synchronize data between instances, therefore it both produces and consumes data from Kafka, additionally the application is consuming a Kafka Topic and transforming and streaming that data into another…
Raystorm
  • 6,180
  • 4
  • 35
  • 62
1
vote
1 answer

Can messages be lost at kafka producer side during deployment?

We are facing a peculiar issue and seeing that when we produce messages to Kafka, it is sometimes not being found at the consumer end. We tried to debug this further and enabled the onSuccess() and onFailure() callbacks. We got that major issue was…
The-Proton-Resurgence
  • 808
  • 1
  • 13
  • 28
1
vote
1 answer

Why async producer is producing messages with linger.ms and batch.size set to a large value and autoFlush is set to false?

I am using spring-kafka 2.2.8 and writing a simple async producer with the below settings: producer config key : compression.type and value is : none producer config key : request.timeout.ms and value is : 10000 producer config key : acks and…
Raj
  • 1,467
  • 3
  • 23
  • 51
1
vote
1 answer

Is there any mechanism in kafka that will reject message if the message has the key , but their is a already message in the kafka with same key?

Is there any mechanism in kafka that will reject message if the message has the key , but their is a already message in the kafka with same key ?
Bravo
  • 8,589
  • 14
  • 48
  • 85
1
vote
1 answer

Why async producer is not waiting for either linger.ms or batch.size to be full when i set them a custom value with autoFlush is set to false?

I am using spring-kafka 2.2.8 and writing a simple async producer with the below settings: linger.ms : 300000, batch.size: 33554431, max.block.ms: 60000. Now i'm creating a KafkaTemplate with autoFlush as false by calling the below…
Raj
  • 1,467
  • 3
  • 23
  • 51
1 2 3
…
99 100