1

I am trying to send 1000's of messages in sequence using Amazon's Kinesis but messages being received by kinesis consumer is not in sequence. Things I have tried :

  1. Used putRecord() method to send with SequenceNumberForOrdering. Achieved ordered sequence with this but throughput is very less.
  2. Used putRecords method to send but not successful.
  3. Used KCL + Amazon's API consumer.
superB
  • 288
  • 2
  • 13

1 Answers1

1

Kinesis maintains sequence within shards, not across all shards in a stream. I am guessing you have more than 1 shard in your stream and hence observing the read records out of sequence. (You can also quickly validate this by repeating #2 with just 1 shard in your stream.)

Quote from here:

please note that Record Sequence Numbers (RSNs) are not globally tracked in the stream and are instead only tracked per shard.

ketan vijayvargiya
  • 5,409
  • 1
  • 21
  • 34
  • Yes it maintains sequence within shards only. But if you are sending many messages at once, it is not maintaining the order. Because of latency issues. Kinesis is only maintaining the order of messages within KinesisQueue but not how producer is sending them. – superB Aug 05 '17 at 13:18
  • what is KinesisQueue? – ketan vijayvargiya Aug 06 '17 at 00:51