Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
1
vote
2 answers

How can I test that I have configured ChainedKafkaTransactionManager correctly in my spring boot service

My spring boot service needs to consume kafka events off one topic, do some processing (including writing to the db with JPA) and then produce some events on a new topic. No matter what happens I cannot have a situation where I have published events…
laurie
  • 265
  • 1
  • 2
  • 9
1
vote
0 answers

How to filter avro events in topic using TopicRecordNameStrategy while consuming Kafka events?

I am using @KafkaListener from Spring Boot to consume avro events from a topic. My subject strategy: TopicRecordNameStrategy If there are multiple avro events like EmployeeCreatedEvent, EmployeeDeletedEvent etc, how can i consume only…
Spartacus
  • 337
  • 4
  • 12
1
vote
1 answer

Consumer is not getting down when processing time of the consumer exceeds more than max.poll.interval.ms

I am trying to do a simple poc on spring kafka with spring boot with version (2.3.7 Release) to achieve how consumer batch process works, and how rebalance works if consumer takes more processing time as i am completely new to this messaging…
MR_K
  • 117
  • 4
  • 17
1
vote
1 answer

How to Handle a Kafka Record with a Class-Level @KafkaListener with no @KafkaHandler for the Record Value

Normally, when we define a class-level @KafkaListener and method level @KafkaHandlers, we can define a default @KafkaHandler to handle unexpected…
Gary Russell
  • 166,535
  • 14
  • 146
  • 179
1
vote
1 answer

How to handle Kafka container lifecycle using spring kafka in Kubernetes multipod deployment

I am using Spring kafka implementation and I need to start and stop my kafka consumer through an REST API. For that i am using KafkaListenerEndpointRegistry…
rajatrj03
  • 13
  • 3
1
vote
2 answers

Is it possible to configure/code a Kafka consumer application for "Exactly Once" failure recovery w/o calling Producer methods?

Is it possible to configure/code a Kafka consumer application to unilaterally implement "Exactly Once Semantics" to handle failure recovery (i.e., resume where left off after a comm failure, etc) independent of producer code (calling KafkaProducer…
sairn
  • 461
  • 3
  • 24
  • 58
1
vote
1 answer

get the unprocessed message count in spring kafka

we are migrating to Kafka, I need to create a monitoring POC service that will periodically check the unprocessed message count in the Kafka queue and based on the count take some action. but this service must not read or process the message,…
shrikant.sharma
  • 203
  • 1
  • 17
  • 37
1
vote
2 answers

Send two Serialized Java objects under one Kafka Topic

I want to implement Kafka Consumer and Producer which sends and receives Java Objects. Full Source I tried this: Producer: @Configuration public class KafkaProducerConfig { @Value(value = "${kafka.bootstrapAddress}") private String…
Peter Penzov
  • 1,126
  • 134
  • 430
  • 808
1
vote
1 answer

Spring kafka - Failed to construct kafka consumer

I'm having an issue with getting my Kafka / confluent spring boot with gradle project up and running. I originally had just a producer in this test project and everything was running well. I then added a Kafka consumer and now I get an exception on…
MetaCoder
  • 368
  • 5
  • 22
1
vote
1 answer

Kafka & Spring Batch - How to read ONLY uncommitted messages from the same topic?

I am working on a small batch with Spring batch and Kafka that reads json data from a Kafka topic, converts it to a Student object, changes a value and sends it back into a Kafka topic. Everything is working fine, but my only problem is that my…
help me code
  • 149
  • 2
  • 13
1
vote
0 answers

Spring Kafka Accumulator Use Case

I am developing a SpringBoot application which consumes events from a Kafka (broker version is 2.6) input topic and produce an event into an output topic. In order to respect some business constraints the component should wait to have at least X…
Giuseppe
  • 307
  • 3
  • 18
1
vote
1 answer

Kafka producer posting messages to secondary cluster

Description of proposed cluster setup 2 Data centres and each having 5 node Kafka cluster Clusters are having the same topics and same producer/consumer instances working with it There is no data replication across the clusters. So data in Cluster…
Mayank Madhav
  • 429
  • 1
  • 7
  • 19
1
vote
2 answers

How to recieve kafka message when testing kafka outbound channel adapter

I'm sending webhook push from a third party app as a message to kafka adpater for further processing by a downstream microservice. The integration flow is as follows @Slf4j @Configuration @AllArgsConstructor(onConstructor = @__(@Autowired)) public…
Anadi Misra
  • 1,925
  • 4
  • 39
  • 68
1
vote
1 answer

Kafka producer with spring kafka template factory implementation

I have a simple rest api (Approach 1) , which produces message sends to kafka cluster using kafka-clients api . Spring boot rest -> producer.send (kafka-clients lib) -> kafka cluster Additionally i also have another implementation (Approach…
Sekhar
  • 961
  • 5
  • 15
  • 27
1
vote
1 answer

Kafka consumer does not fetch new records when using topic pattern and large messages

I hope someone of you can help me. I'm using spring boot 2.3.4 with spring kafka 2.5.6. I recently had to reset an offset and saw some strange behavior. We consumed the messages, but after every X (variating) messages we had a timeout of 10 seconds…
Kraennix
  • 11
  • 1
  • 3
1 2 3
…
99 100