Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
11
votes
2 answers

Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 1001: org.apache.kafka.common.errors.DisconnectException

I am using spring kafka and facing some errors Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 1001: org.apache.kafka.common.errors.DisconnectException. my consumer producer code is given @EnableKafka @Configuration public…
Edayan
  • 537
  • 1
  • 7
  • 17
11
votes
1 answer

What happens to a Kafka consumer group when all consumers are removed

I have a service with an autoscaler and each instance needs to be in a separate consumer group, I've achieved it by making a random consumer group name group-id: my-service-${random.uuid}. I want to know what happens to a consumer group if all…
Michael Dz
  • 3,655
  • 8
  • 40
  • 74
11
votes
1 answer

Kafka Acknowledgment vs Kafka commit

What is the difference between Kafka Acknowledgment and Kafka consumer commitSync () Both are used for purpose of manual offset management, and hope both works Synchronously. Please assist
Arpit Saklecha
  • 123
  • 1
  • 7
11
votes
1 answer

Spring Boot Kafka Startup error "Connection to node -1 could not be established. Broker may not be available."

I am trying to start Spring-Kafka with Spring Boot 2.1.7.RELEASE on localhost with Java 12. Getting the Error: "org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=inter] Connection to node -1 could not be established.…
Philip Schikora
  • 283
  • 1
  • 3
  • 9
11
votes
2 answers

How to skip corrupt (non-serializable) messages in Spring Kafka Consumer?

This question is for Spring Kafka, related to Apache Kafka with High Level Consumer: Skip corrupted messages Is there a way to configure Spring Kafka consumer to skip a record that cannot be read/processed (is corrupt)? I am seeing a situation where…
Shankar
  • 2,625
  • 3
  • 25
  • 49
11
votes
2 answers

Spring boot spring.kafka.bootstrap-servers not getting picked up by consumer config

I'm trying to use connect a spring boot project to kafka . In my application.properties file I have the following…
lights
  • 1,034
  • 1
  • 8
  • 22
11
votes
2 answers

Managing Kafka Topic with spring

We are planning to use Kafka for queueing in our application. I have some bit of experience in RabbitMQ and Spring. With RabbitMQ and Spring, we used to manage queue creation while starting up the spring service. With Kafka, I'm not sure what…
Thiru
  • 2,541
  • 4
  • 25
  • 39
11
votes
2 answers

Spring Kafka Auto Commit Offset In Case of Failures

I am using Spring Kafka 1.2.2.RELEASE. I have a Kafka Listener as consumer that listens to a topic and index the document in elastic. My Auto commit offset property is set to true //default. I was under the impression that in case there is an…
rishi
  • 173
  • 1
  • 1
  • 7
11
votes
1 answer

How to create separate Kafka listener for each topic dynamically in springboot?

I am new to Spring and Kafka. I am working on a use case [using SpringBoot-kafka] where in users are allowed to create kafka topics at runtime. The spring application is expected to subscribe to these topics pro-grammatically at runtime. What i know…
Yashwanth
  • 129
  • 1
  • 7
10
votes
0 answers

EmbeddedKafka w/ ContainerTestUtils.waitForAssignment throws: Expected 1 but got 0 partitions

We have an integration test where we use EmbeddedKafka and produce a message to a topic, our app processes that message, and the result is sent to a second topic where we consume and assert the output. In CI this works maybe 2/3 of the time, but we…
archonic
  • 375
  • 4
  • 15
10
votes
1 answer

Spring Boot 2.3.2.RELEASE - Micrometer - KafkaMetrics - Failed to bind meter logs

I am using /actuator/prometheus endpoint for its kafka_consumer_* metrics. Upgrading from Spring Boot 2.3.1.RELEASE to 2.3.2.RELEASE, is showing me a lot of these "extra" logs - whenever there is a failure to…
jumping_monkey
  • 5,941
  • 2
  • 43
  • 58
10
votes
0 answers

What is the cleaner and efficient way of implementing health check for Kafka in my spring boot application?

I have a spring boot (2.1.6) application that both consumes and produces messages to a (organization wide) common kafka instance. I am trying to implement health checks for this kafka broker in my application using spring actuator and I am facing a…
ishan
  • 1,202
  • 5
  • 24
  • 44
10
votes
1 answer

How to auto configure programmatically?

I have a spring boot kafka application. My brokers are recycled every few days. The old brokers are deprovisioned and new brokers are provisioned. I have a scheduler which is checking for brokers every few hours. I would like to make sure as soon…
s7vr
  • 73,656
  • 11
  • 106
  • 127
10
votes
2 answers

Spring KafkaListener: How to know when it's ready

I have a simple Spring Boot application which reads from Kafka and writes to Kafka. I wrote a SpringBootTest using an EmbeddedKafka to test all that. The main problem is: Sometimes the test fails because the test sends the Kafka message too early.…
konse
  • 885
  • 1
  • 10
  • 21
10
votes
2 answers

How to filter Kafka messages before consumer consume in spring Kafka

I am using spring Kafka in my project and I want to filter messages before consumer consumes based on key and value. Is it possible?
app
  • 733
  • 5
  • 15
  • 27
1 2 3
…
99 100