Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
8
votes
1 answer

EmbeddedKafka how to check received messages in unit test

I created a spring boot application that sends messages to a Kafka topic. I am using spring spring-integration-kafka: A KafkaProducerMessageHandler is subscribed to a channel (SubscribableChannel) and pushes all messages received to…
dermoritz
  • 12,519
  • 25
  • 97
  • 185
8
votes
1 answer

High memory consumption on kafka consumer

Recently I found that kafka consumers require a lot of ram. For tests I've just started locally a single-threaded consumer that listens a single topic.Topic has 4 partitions. Kafka has only one broker. From producer I sent only 10 small messages (it…
rvit34
  • 1,967
  • 3
  • 17
  • 33
7
votes
5 answers

Kafka consumer error Cancelled in-flight API_VERSIONS request with correlation id 1 due to node -1 being disconnected

I have the consumer config as follows package com.example.kafka.config; import org.apache.kafka.clients.consumer.ConsumerConfig; import org.apache.kafka.common.serialization.StringDeserializer; import…
Tom
  • 371
  • 2
  • 6
  • 15
7
votes
4 answers

Cannot convert from [java.lang.String] to [com.example.demo.User]

I'm working on Spring Boot and Apache Kafka - trying to have a user defined configurations - org.springframework.kafka.listener.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message Endpoint handler…
PAA
  • 1
  • 46
  • 174
  • 282
7
votes
1 answer

Randomly generated group id for Kafka Consumer

We have an application that uses spring kafka to read messages. It is necessary that each instance of the application has a unique groupId, as well as reset it and get a new one on restart. The GroupId is randomly generated via ${random.uuid}. Is…
Фёдор Антонов
  • 95
  • 1
  • 4
7
votes
1 answer

NoClassDefFoundError: scala/math/Ordering with spring-kafka-test 2.5.7

I have a small library that provides a JUnit 5 extension around EmbeddedKafkaBroker, which tries to be more performant than just using @DirtiesContext by instead resetting the offsets to latest between tests instead of throwing away the whole…
mrusinak
  • 1,002
  • 1
  • 12
  • 22
7
votes
2 answers

Spring Kafka don't respect max.poll.records with strange behavior

Well, I'm trying the following scenario: In application.properties set max.poll.records to 50. In application.properties set enable-auto-commit=false and ack-mode to manual. In my method added @KafkaListener, but don't commit any message, just…
userdevexp
  • 169
  • 2
  • 7
7
votes
2 answers

How to use @Transactional with @KafkaListener?

Is there any possibility to use declarative tx management (via @Transactional) with @KafkaListener annotated method ? I would like to use it in order to, for example, define separate tx timeout per listener. My setup is as…
Łukasz Torzyński
  • 141
  • 1
  • 3
  • 6
7
votes
1 answer

Disabling Kafka Listeners for a particular Spring Boot test

How to disable @KafkaListener instances in @SpringBootTest tests in applications with Spring Boot (2.2+) and Spring Kafka (2.4+)? The goal is to disable Kafka listeners in particular tests, so that such tests can run without starting an embedded…
Eugene Khyst
  • 9,236
  • 7
  • 38
  • 65
7
votes
1 answer

spring kafka offset increment even auto commit offset is set to false

I am trying to implement manual offset commit for the messages received on kafka. I have set the offset commit to false, but the offset value keeps on increasing. Not sure what is the reason. Need help resolving the issue. Below is the…
Tushar Banne
  • 1,587
  • 4
  • 20
  • 38
7
votes
2 answers

how to compress data in producers when using spring kafka

I am currently sending data using spring-kafka like this: val json = objectWriter.writeValueAsString(obj) kafkaTemplate.send(topic, json) How do i tell KafkaTemplate to use compress the json using snappy before sending?
pdeva
  • 43,605
  • 46
  • 133
  • 171
7
votes
1 answer

Reset EmbeddedKafka After Every Test Method

I am writing a test class that has multiple methods that require Kafka. Each one requires different broker properties, so I want a separate instance of EmbeddedKafka for each. Doing @EmbeddedKafka( partitions = 20, topics = {"topic"}, …
Prashant Pandey
  • 4,332
  • 3
  • 26
  • 44
7
votes
0 answers

ZooKeeper session expired in tests

I'm using the EmbeddedKafka for testing my module with the following annotation confiugration: @ExtendWith(SpringExtension.class) @SpringBootTest @TestPropertySource(locations = "classpath:test.properties") @EmbeddedKafka(partitions = 1, topics…
user1028741
  • 2,745
  • 6
  • 34
  • 68
7
votes
1 answer

problem path for truststore inside docker with spring boot and kafka

i have technical problem trying solving when deploying my app spring boot app with docker container. org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: Failed to load SSL keystore…
DarkVision
  • 1,373
  • 3
  • 20
  • 33
1 2 3
…
99 100