Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
1
vote
1 answer

Manual Acknowledge (commit) kafka messages with ReplyingKafkaTemplate

I'm using ReplyingKafkaTemplate.sendAndReceive() to send and receive a message correlated by correlation id. The use case have many topics on the way, and I need to manual acknoledge (commit) consumed messages offsets. So far so good, this is done…
Fernando Ruscitti
  • 33
  • 5
1
vote
1 answer

Numerous Kafka Producer Network Thread generated during data publishing, Null Pointer Exception Spring Kafka

I am writing a Kafka Producer using Spring Kafka 2.3.9 that suppose to publish around 200000 messages to a topic. For example, I have a list of 200000 objects that I fetched from a database and I want to publish json messages of those objects to a…
biswas
  • 107
  • 1
  • 11
1
vote
0 answers

Kafka Listener - Configure Interceptor in Spring Boot?

I am trying to configure the Confluent - ConsumerTimestampsInterceptor to support the Confluent KAFKA Replication and have configured Java spring boot application as mentioned below application.properties #consumer timestamp…
One Developer
  • 99
  • 5
  • 43
  • 103
1
vote
2 answers

Avro GenericRecord deserialization not working via SpringKafka

I'm trying to simplify my consumer as much as possible. The problem is, when looking at the records coming in my Kafka listener: List incomingRecords the values are just string values. I've tried turning specific reader to true and…
Ryan
  • 1,102
  • 1
  • 15
  • 30
1
vote
0 answers

Spring boot kafka producer send large mex

In some solutions reported on StackOverflow, we are told to increase the MAX_REQUEST_SIZE_CONFIG. In my case it does not help, because the broker cannot be changed: it is set to receive 1 MB messages. Can we send a message with a larger size? We…
DrBomber
  • 25
  • 7
1
vote
1 answer

Pause/Start Kafka Stream processors in Spring boot

I'm going to implement a Circuit breaker pattern for messages. Basic requirement is that if microservice cannot publish messages to publishing topic it should stop accepting messages from other Kafka topics. When the publishing topics become…
Keaz
  • 955
  • 1
  • 11
  • 21
1
vote
1 answer

Spring boot application stop serving traffic while Kafka consumer rebalancing

I'm running Spring boot applications in k8s cluster with Kafka. during a rolling update or scaling my services, some of them rebalanced which is ok since consumers are being added or removed, but this causes the service whos rebalancing to stop…
guy dadon
  • 282
  • 1
  • 2
  • 13
1
vote
1 answer

How can I use Custom Partition with KafkaSender.send method()?

I have made a custom Partitioner class that extends default Partitioner. Problem : I want to add this custom Partitioner in KafkaSender.send method() KafkaSender.send method() code : sender.send(Flux.just(SenderRecord.create(new…
rushabh
  • 59
  • 3
1
vote
1 answer

Is there a way to update the number of concurrency in ConcurrentMessageListenerContainer?

I developing an application which using the spring-kafka package and changing the amount of data it handled in run time. I want to let the system the ability to change in run time the number of concurrency depend on the pressure the system feel…
Lupidon
  • 599
  • 2
  • 17
1
vote
2 answers

How to do integration testing of KStream topology using spring-boot EmbeddedKafka?

I have a simple spring-boot KStream topology that transforms a string from lowercase to uppercase. I want my integration test to launch an embedded kafka, and then test the topology. I would like to know if it possible to write integration tests…
tintin
  • 5,676
  • 15
  • 68
  • 97
1
vote
2 answers

Spring Cloud Kafka: Can't serialize data for output stream when two processors are active

I have a working setup for Spring Cloud Kafka Streams with functional programming style. There are two use cases, which are configured via application.properties. Both of them work individually, but as soon as I activate both at the same time, I get…
Bennett Dams
  • 6,463
  • 5
  • 25
  • 45
1
vote
1 answer

Cassandra trigger to Kafka on TTL expire

I'm trying to implement a trigger on Cassandra to catch delete operations on TTL expire by implementing public Collection augment(Partition partition) on ITrigger interface. But event is not triggering to kafka on ttl expire.
Rishabh Agarwal
  • 21
  • 2
1
vote
0 answers

ClassCastException: class java.lang.String cannot be cast to class SaleResponseFactory

I want to implement Kafka Consumer and Producer which sends and receives Java Objects. Full Source I tried this: Producer: @Configuration public class KafkaProducerConfig { @Value(value = "${kafka.bootstrapAddress}") private String…
Peter Penzov
  • 1,126
  • 134
  • 430
  • 808
1
vote
2 answers

Spring-kafka with spring boot actuator error - Caused by: java.lang.ClassNotFoundException: org.springframework.kafka.core.ProducerFactory$Listener

This is wired my spring boot application was working fine with spring-kafka dependency. For application monitoring with prometheus as soon I add spring-boot-starter-actuator dependency, my application blows up with exception. If I remove the…
Dheeru
  • 31
  • 1
  • 6
1
vote
1 answer

Unable to create Kafka DealLetterTopic using DeadLetterPublishingRecoverer

I am trying to create a dead letter topic using DeadLetterPublishingRecoverer but unable to create it. Below are the beans that i am defining. But i see in the logs that in the ProducerConfig, i dont see the property allow.auto.create.topics. Its…
user1945064
  • 199
  • 3
  • 4
  • 18
1 2 3
…
99 100