We have three containers connecting to Kinesis stream as consumers using Spring cloud AWS kinesis binder. We have used consumer groups for load balancing across three containers. The requirement is that containers share the load and it shall be possible to distribute the load evenly.
- All containers use the same application config from config server
Currently we have configuration like below.
spring:
cloud:
stream:
bindings:
MyStream:
group: my-group
destination: stream-1
content-type: application/json
All containers are consuming same message, but at the different times (having difference of 5 to 10 mins)
As per the documentation,
Static shard distribution within a single consumer group It is possible to evenly distribute shard across all instances within a single consumer group. This done by configuring:
spring.cloud.stream.instanceCount= to number of instances
spring.cloud.stream.instanceIndex= current instance’s index
As we use the same config server, can you please help how to make sure that load balancing is achieved.