Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
1
vote
3 answers

Number of consumers in a single Kafka comsumer-group

I want to consume 3 topics, with 4 partitions each, using a single consumer group. Should the consumer group contain 3, 4, or 12 consumers? I want to achieve optimal consumption.
avivas
  • 11
  • 2
1
vote
1 answer

How can I effectively bind my @KafkaListener to ConcurrentKafkaListenerContainerFactory?

I hit this scenario which appears strange to me: So basically I have defined two @KafkaListener in one class: @KafkaListener(id = "listener1", idIsGroup = false, topics = "data1", containerFactory = "kafkaListenerContainerFactory") public void…
yifei
  • 561
  • 5
  • 18
1
vote
0 answers

Spring kafka with ssl, producer error "Duplicate key"

I have a Kafka server with SSL. When producing messages to it, error occurs: 2020-05-29 19:19:27.677 [kafka-producer-network-thread | producer-1] ERROR o.a.k.c.producer.internals.Sender - [Producer clientId=producer-1] Uncaught error in kafka…
cosimoth
  • 167
  • 10
1
vote
1 answer

Garbage value published as key in kafka topic

I am using Long serializer for the keys and String serializer for value , after posting the message to the kafka topic when we retrieved the message and along with key are seeing key as some garbage value like below ^@^@^@^AÏÃ<9a>ò is there…
Bravo
  • 8,589
  • 14
  • 48
  • 85
1
vote
1 answer

Post to topic and Manual commit in spring kafka

We are working on a requirement where the Spring-kafka consumer is committing to a topic and then should manually commit the offset. One of the scenario that we are thinking is what happens when the consumer fails between the time that it submits…
codebuilder
  • 23
  • 1
  • 5
1
vote
1 answer

org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition - keep appearing

we are using spring, kafka(spring kafka 2.4.5, spring cloud stream 3.0.1), openshift. we have the below configuration. Multiple broker/topic with each topic has 8 partitions with replication factor as 3, multiple spring boot consumer. we are…
Ram
  • 47
  • 1
  • 5
1
vote
1 answer

KafkaTemplate.send(key,value,topic) with Custom Partitioner?

I saw and implemented the method KafkaTemplate.send(TOPIC,message) with default partitioner class. But here, I am not passing keys. I have a simple custom partitioner class and I also wanna send to kafka server like KafkaTemplate(TOPIC,key,message)…
Nafiul Alam Fuji
  • 407
  • 7
  • 17
1
vote
1 answer

Multiple spring cloud stream application running together

I referred to the example posted here. I am trying to run Multiple spring cloud stream application together. Here the output of first is given as input to other. Below is what I am trying to do. @Bean public Function
Anupam Gupta
  • 1,591
  • 8
  • 36
  • 60
1
vote
1 answer

Kakfa Consumer Commit on my own offset id not working - commitSync(final Map offsets)

I'm fetching a bunch of messages (say 100) from a topic using poll(). I have set auto-commit to false and max.poll.records to 100 in my configuration. I will consume 10 messages from 100 messages which I received for my processing. Hence I want the…
user3373917
  • 55
  • 6
1
vote
1 answer

How to enable Stateful Retry using spring-cloud-stream-binder-kafka and RetryTemplate?

I am wondering if there is a way to enable Stateful RetryTemplate using spring-cloud-stream-binder-kafka. I noticed that there is a constructor RetryingMessageListenerAdapter(MessageListener messageListener, RetryTemplate retryTemplate,…
jskim1991
  • 40
  • 1
  • 5
1
vote
1 answer

SSL.keystore.location can't find JKS file in my Kubernetes secrets mount

I have created a secret for my JKS file under volume mount /etc/secrets/keystore. I am accessing my JKS file path as an environment variable where ssl.keystore.location gets resolved as file:///etc/secrets/keystore/ssl.jks. But I get exception from…
victor perfect
  • 31
  • 2
  • 6
1
vote
0 answers

java.lang.ClassNotFoundException: org.springframework.kafka.listener.RecordInterceptor

I have my spring boot kafka application with maven and I have updated the older versions with the latestone for spring-boot-starter-parent and spring-kafka as below. spring-boot-starter-parent from 2.1.7.RELEASE --to-->…
Mayuri
  • 11
  • 4
1
vote
1 answer

How can i extend spring-kafka's '@KafkaListener' annotation to create my own annotation with limited attributes?

I've been using '@KafkaListener' at method level to create consumers. Now, I'm trying to create my own custom annotation by extending '@KafkaListener' and limit the no of attributes (for example, because of some reasons, I don't want to expose…
Raj
  • 1,467
  • 3
  • 23
  • 51
1
vote
1 answer

Shutdown kafka consumer after processing messages

I am using @KafkaListener(topics = "${topic}") to consume messages from a topic in a spring-boot application and I need this running periodically. spring-kafka version is 2.2.4.RELEASE. One way to achieve this could have been batching every 6 hours…
Rupesh Ranjan
  • 13
  • 3
1
vote
1 answer

ZooKeeperClientTimeoutException when using @EmbeddedKafka

Once I add @EmbeddedKafka to an integration test I see the following error: Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedKafka': Invocation of init method failed; nested exception is…
Andras Hatvani
  • 4,346
  • 4
  • 29
  • 45
1 2 3
…
99
100