An Apache Kafka Binder for Spring-Cloud Streams.
Documentation: https://cloud.spring.io/spring-cloud-stream-binder-kafka/spring-cloud-stream-binder-kafka.html
Related:
Questions tagged [spring-cloud-stream-binder-kafka]
472 questions
1
vote
1 answer
Kafka record headers won’t be forwarded to changelog topics
I need the headers in every topic for object mapping purposes.
Currently when I add a header to a record in a processor, then the header will be present in the target output topic, but not in changelog topics used by state stores.
How can the…
Andras Hatvani
- 4,346
- 4
- 29
- 45
1
vote
1 answer
How to use the output binding of spring-cloud-stream in unit tests?
I'm using spring cloud stream to modify a Kafka Topic and write the resulting data with ".toTable()" to a table. In the application.yaml I will set the input and output bindings.
This works fine on the Kafka cluster but not with my current test…
manuelgr
- 498
- 1
- 8
- 26
1
vote
0 answers
Configure spring boot app with multiple Kafka consumer clusters each with different SSL configs?
I have a single spring boot application which needs to consume data from topics belonging to two different Kafka clusters having separate SSL configs(keystore and trusstore files & locations).
Is it possible to configure this in…
Vaibhav
- 45
- 6
1
vote
0 answers
Kafka Stream aggregate function sinking each joined record in an incremental way, instead of a single list of aggregated records
I’m having some issues with my Kafka Streams implementation in production. I’ve implemented a function that takes a KTable and a KStream, and yields another KTable with aggregated results based on the join of these two inputs. The idea is to iterate…
hedz
- 111
- 1
- 7
1
vote
1 answer
Missing admin properties in KafkaBinderConfigurationProperties
The KafkaBinderConfigurationProperties class has dedicated properties for the producer and consumer. However, there is no admin properties.
While comparing with KafkaProperties which has the dedicated properties for producer, consumer and admin.
I…
Yi Liu
- 83
- 4
1
vote
0 answers
Spring boot Application is not Shutdown if not able to connect to Stream binder
I am connecting kafka with Spring boot application using spring cloud stream Functinal style programming model.
I need to stop the application if the application is not able to connect to kafka binder.
Following are the properties I am using
Spring…
Awais Yousaf
- 23
- 7
1
vote
0 answers
kafka cloud stream consumer condition
When i migrated spring-cloud-starter-stream-kafka to the version 3.2.5 i had many annotations that are duplicated, that i'm correcting
old consumer:
@StreamListener(value = KafkaStreams.INPUT_CATALOG, condition = "headers['catalog-code'] ==…
Aymen Kanzari
- 1,765
- 7
- 41
- 73
1
vote
1 answer
transactional.id.expiration.ms doesn't expire producer ids when set manually
I want to enforce a certain behaviour that's why I've set transactional.id.expiration.ms: 1000 via strimzi and verified that it will be picked up by the broker too.
My expectation is that the producer ids expire after 1 second too, but this doesn't…
Andras Hatvani
- 4,346
- 4
- 29
- 45
1
vote
1 answer
Spring-Kafka BatchInterceptor don't work as expected
I have spring-cloud-stream project that use kafka binder.
Application consumes messages in batch mode. I need to filter consumed records by specific header. In this case i use BatchInterceptor:
@Bean
public…
Vichukano
- 99
- 8
1
vote
0 answers
Continuously Getting "Found no committed offset for partition 'topic-name-x' " for new consumer and consumer group
I am using "spring-cloud-stream-binder-kafka" for my consumer, having AVRO topic.
it's new consumer with new consumer group. After running the application I am getting this log "Found no committed offset for partition 'topic-name-x'". I read that it…
APK
- 155
- 1
- 15
1
vote
0 answers
Spring cloud stream kafka setting ssl properties programmatically
spring.cloud.stream.kafka.binder.configuration.ssl.truststore.location=/Users/truststore.jks
spring.cloud.stream.kafka.binder.configuration.ssl.keystore.location=/Users/keystore.jks
spring.cloud.stream.kafka.binder.configuration.ssl.truststore.passwo…
Vibhor Pareek
- 21
- 2
1
vote
1 answer
Accessing retry attempt number in spring cloud stream kafka transaction
I need to access the retry attempt number in spring cloud stream kafka transactional retry so that for a particular exception, based on the retry attempt number i can post the outcome to different topic
DreamCoder
- 23
- 5
1
vote
0 answers
Kafka Streams Health Indicator with a restoring state store
The default setup with kafka-streams and spring-boot-actuator includes adding the state of the stream threads into the health check endpoint. This is great, but if my application is using a state store, and that store is still in the process of…
M21B8
- 1,867
- 10
- 20
1
vote
2 answers
stream bridge provisions a topic in Kafka with the binding name for input binding of a function
I have a function publish for which I've defined the binding too
@Bean
public Function, Message>> publish() {
// some magic here
}
the config…
Anadi Misra
- 1,925
- 4
- 39
- 68
1
vote
0 answers
Spring Cloud Kafka Stream Multibinder BeanCreationException: Error creating bean with name 'kafkaStreamsFunctionProcessorInvoker'
I am trying to create a spring boot application using spring cloud kafka stream which reads input from kafka cluster 1 and send it to kafka cluster 2 using single kafkastream application.
I am getting following exception during…
Ketan Yadav
- 11
- 2