1

I have two types of events that I plan to put in kinesis. Event A has high frequency so I need the batch-size as 100 and maxRecordAge as 1 minute. Event B is less often and I would like to keep the batch-size as 1 as I want to process event as soon as possible. I plan to keep both the events in same kinesis but on different shards. Is there a way to configure batch-size based on shards? I am using Kinesis lambda trigger to trigger the lambda.

aran
  • 10,978
  • 5
  • 39
  • 69
dontez
  • 11
  • 1

1 Answers1

1

No, there is no way to define the batch size per shard. Batchsize configuration is defined on the stream level.

There are two possibilties to handle this scenario:

  1. Either you have two lambda functions as acting as consumers, each one with different batch size, and based on the message key, you can filter the messages for high frequency/low frequency data

  2. Split the data into two seperate streams and add a lambda consumer to each stream.

Nishu Tayal
  • 20,106
  • 8
  • 49
  • 101