Questions tagged [spring-cloud-stream-binder]
51 questions
0
votes
1 answer
Spring cloud stream kafka timestamp
I am using spring cloud streams for consuming kafka. I need to get the time the event has been published to the queue from the publisher(publisher time not the broker ingestion time)
I can see below information in…
ging
- 219
- 7
- 20
0
votes
1 answer
When rabbitMQ is down in spring boot, startup time increases due to connection retries
I have a spring boot micro service application and I am using spring-cloud-stream-binder-rabbit
All my rabbiqMq configurations are working fine but if rabbitMq goes down, consumers keeps attempting to fetch the connection indefinitely increasing the…
Naren
- 1
- 2
0
votes
1 answer
KCL doesn't PUT or GET and items from DynamoDB checkpoint or locks tables
I implemented reading messages from Kinesis using KCL, but when i check DynamoDB tables, i found that only group table have the list of shards, but checkpoint table and locks table doesn't have any items:
Any idea why KCL doesn't PUT or GET items…
mibrahim.iti
- 1,928
- 5
- 22
- 50
0
votes
0 answers
KCL Encountered an exception while renewing a lease
I implement reading from AWS/Kinesis stream using KCL, but after read many messages from shards, suddenly i see this error in server logs:
ERROR 1 --- [oordinator-0000] c.a.s.kinesis.leases.impl.LeaseRenewer : Encountered an exception while…
mibrahim.iti
- 1,928
- 5
- 22
- 50
0
votes
0 answers
KCL reader AWS instances goes down quickly due to heap memory exception
I wrote next implementation to read messages from AWS Kinesis and deployed it on AWS ECS:
Application.java
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import…
mibrahim.iti
- 1,928
- 5
- 22
- 50
0
votes
0 answers
Spring cloud : 2021.0.4 : Cloud Stream Partitioning is failing for RabbitMQ
In Cloud Stream "2021.0.4", we are facing issue: Routing to destination is failing when Partitioning is enabled for the RabbitMQ. Binder always throwing "Partition key can't be null". This does not occur when we rollback to 2021.0.3. Please note…
Siva Kumar
- 1
- 1
0
votes
1 answer
Confusing messaging regarding the future of Spring Cloud Schema Registry
Here, it's explained that "starting with 3.2.x release of Spring Cloud Stream, we stopped shipping the schema registry clients (artifacts such as spring-cloud-schema-registry-client) as part of Spring Cloud Stream." If this is true, then can someone…
Keith Bennett
- 733
- 11
- 25
0
votes
0 answers
Spring Cloud Stream Kafka Binder Async Producer messages getting lost if broker goes down
I am using Spring Cloud Stream Kafka Binder to produce message into Kafka. I kept producer sync to false, enabled error channel for producer, unclean leader election true on server…
Ranjit Meher
- 41
- 4
0
votes
1 answer
Spring Cloud Stream with RabbitMQ Streams
I'm trying to use the "new" Streams plugin for RabbitMQ with my spring-cloud-stream project using "functional programming model".
I have set up my application.yaml like this:
spring:
rabbitmq:
listener:
type: stream
stream:
…
bsgrd
- 633
- 1
- 9
- 26
0
votes
1 answer
How to produce a message record which can be consumed by a spring cloud stream consumer?
From the producer side, I'm using Kinesis's PutRecord API to produce messages to the stream. From the consumer side, I'm using spring-cloud-stream-binder integration for kinesis.
I have adapted the code for building the PutRecordRequest object from…
Augustine Theodore
- 45
- 2
- 6
0
votes
0 answers
How to make different instances of consumers in the same consumer group consume different shards of the same kinesis stream?
I'm following the example given in spring-cloud-stream-samples with the following modifications.
application.yml
spring:
cloud:
stream:
instanceCount: 2
bindings:
produceOrder-out-0:
destination: test_stream
…
Augustine Theodore
- 45
- 2
- 6
0
votes
1 answer
How do I acknowledge / requeue with cloud stream sqs binders
I am writing an application to consume messages from queue. I am able to successfully bind the sqs and receive the messages. However, when I want to requeue the message, I am using as follows.
message.getHeaders().get(AwsHeaders.ACKNOWLEDGMENT,…
Mansoor
- 1,157
- 10
- 29
0
votes
1 answer
How do you pass Kinesis headers to Spring Cloud Stream functions in batch consumer mode?
I am in the process of converting our existing stream processing code that currently uses the annotation-based programming model to use Spring Cloud Function instead. With consumer batch mode enabled, how can I pass the AWS Kinesis checkpointer to…
Keith Bennett
- 733
- 11
- 25
0
votes
1 answer
spring cloud stream - gcp pubsub binder maxFetchSize?
try to use spring cloud gcp binder library polling messages from GCP pubsub topic.
ref from streaming-vs-polled-input. use spring.cloud.stream.gcp.pubsub.default.consumer.maxFetchSize poll get N message (maxFetchSize's value) at a time, but even i…
tHe0rAL
- 1
0
votes
0 answers
Spring cloud stream binder gives "Failed to create consumer binding" when connecting to google pubsub
Assume that we have a pubsub topic A in Project A.
I am running a spring boot application in project B.
It uses a Service Account - test-sa-projectB.
The application is set up in the below way to consume the data from topic…
Anudeep Kota
- 89
- 1
- 1
- 3