Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
8
votes
2 answers

Spring-Kafka vs. kafka-clients directly

I have an application based on spring-boot that needs to consume/produce events on Kafka. I am hesitating on the library choice. It seems to be simple to use Kafka-clients directly without having to manage the compatibility matrix between…
brianbro
  • 4,141
  • 2
  • 24
  • 37
8
votes
1 answer

How can I map incoming headers as String instead of byte[] in my Spring Cloud Stream project?

I have a simple Spring Cloud Stream project using Spring Integration DSL flows and using the Kafka binder. Everything works great, but message header values coming from Kafka arrive as byte[]. This means that my SI @Header parameters need to be of…
Gabriel Bauman
  • 2,270
  • 1
  • 22
  • 36
8
votes
2 answers

Kafka Stream Exception: GroupAuthorizationException

I'm developing a Kafka-Stream application, which will read the message from input Kafka topic and filter unwanted data and push to output Kafka topic. Kafka Stream Configuration: @Bean(name =…
Bharathiraja S
  • 679
  • 4
  • 12
  • 26
8
votes
1 answer

In kafka, When producing message with transactional, Consumer offset doubled up

I'm making project using springboot 2, kafk 2.2.0, spring-kafka 2.2.5 I maked kafka exactly once environment and message producing and consuming was well. BUT kafka-consumer-groups.sh saied like this. TOPIC PARTITION CURRENT-OFFSET …
LKC
  • 131
  • 9
8
votes
2 answers

How can I create many kafka topics during spring-boot application start up?

I have this configuration: @Configuration public class KafkaTopicConfig { private final TopicProperties topics; public KafkaTopicConfig(TopicProperties topics) { this.topics = topics; } @Bean public NewTopic…
ip696
  • 6,574
  • 12
  • 65
  • 128
8
votes
2 answers

Transactional Producer vs Just Idempotent Producer Java (Exception OutOfOrderSequenceException)

I use spring-kafka with idempotent producer configuration: these are my configuration props: Properties props = new Properties(); props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, Joiner.on(",").join(appProps.getBrokers())); …
Rakesh Subramanian S
  • 81
  • 1
  • 3
8
votes
1 answer

Kafka broker node goes down with "Too many open files" error

We have a 3 node Kafka cluster deployment, with a total of 35 topics with 50 partitions each. In total, we have configured the replication factor=2. We are seeing a very strange problem that intermittently Kafka node stops responding with…
Ankit Singhal
  • 81
  • 1
  • 1
  • 3
8
votes
1 answer

Spring Kafka MessageListenerContainer

I am seeing spring Kafka code and I have some doubts: If we are using 1 @kafkaListener with 2 topics then spring Kafka creates a single MessageListenerContainer. And if I use separate @kafkaListener for each topic then 2 MessageListenerContainer…
AlwaysLearning
  • 133
  • 1
  • 8
8
votes
0 answers

Spring embedded kafka fails when I'm running tests

I have a problem with spring embedded kafka which I want to use to test my kafka sender/receiver. When I try to run my tests using: @RunWith(MockitoJUnitRunner.class) @SpringBootTest @DirtiesContext public class myTestClass { @ClassRule …
Tomek
  • 81
  • 1
  • 2
8
votes
1 answer

@KafkaListener concurrency multiple topics

I want to create a concurrent @KafkaListener which can handle multiple topics each with different number of partitions. I have noticed that Spring-Kafka only initializes one consumer per partition for the topic with most partitions. Example: I have…
kkflf
  • 2,435
  • 3
  • 26
  • 42
8
votes
2 answers

ClassNotFoundException with Kafka consumer

I have a Kafka consumer application, written with Spring Boot 2.0.2. When I receive the message in my listener, I get the following error: Caused by: org.springframework.messaging.converter.MessageConversionException: failed to resolve class name.…
Joe P
  • 499
  • 1
  • 4
  • 11
8
votes
2 answers

Spring Boot / Kafka Json Deserialization - Trusted Packages

I am just starting to use Kafka with Spring Boot & want to send & consume JSON objects. I am getting the following error when I attempt to consume an message from the Kafka topic: org.apache.kafka.common.errors.SerializationException: Error…
hexkid
  • 574
  • 1
  • 4
  • 13
8
votes
1 answer

How to set timeout for onFailure event (Spring, Kafka)?

I'm trying to implement an asynchronous REST method of sending a message to Kafka in Spring MVC. Everything works, but when the server is unavailable, the onFailure event is processed for a long time. How to limit the response time in…
V. Perfilev
  • 438
  • 1
  • 7
  • 18
8
votes
1 answer

Spring Cloud Kafka Stream Unable to create Producer Config Error

I have two Spring boot project with Kafka-stream dependencies, they have exactly same dependencies in gradle and exactly same configurations, yet one of the project when started logs error as below 11:35:37.974 [restartedMain] INFO …
no name
  • 522
  • 4
  • 13
8
votes
2 answers

Adding custom header using Spring Kafka

I am planning to use the Spring Kafka client to consume and produce messages from a kafka setup in a Spring Boot application. I see support for custom headers in Kafka 0.11 as detailed here. While it is available for native Kafka producers and…
Annu
  • 145
  • 1
  • 3
  • 10
1 2 3
…
99 100