0

When using Kinesis Data Streams, it's possible to use the enhanced fan-out feature?

DynamoDB Streams API says:

The DynamoDB Streams API is intentionally similar to that of Kinesis Streams, a service for real-time processing of streaming data at massive scale. In both services, data streams are composed of shards, which are containers for stream records. Both services' APIs contain ListStreams, DescribeStream, GetShards, and GetShardIterator actions. (Even though these DynamoDB Streams actions are similar to their counterparts in Kinesis Streams, they are not 100% identical.)

Is it possible to use enhanced fan-out with DynamoDB Streams? Is there any documentation or code samples that talks about this use?

It looks like the answer might be no because enhanced fan-out requires the SubscribeToShard method which wasn't listed above.

stephenbez
  • 5,598
  • 3
  • 26
  • 31

1 Answers1

2

Yes, DynamoDb streams does not support the fancy enhanced fan-out. Think of Kinesis Data Streams = dynamoDb streams 2.0. So DynamoDb streams wont be changing anytime soon.

DynamoDb streams was designed primarily as oplog to notify a "single" consumer (two at most). With the question in mind, I assume you want to have multiple consumers with dedicated capacities connecting to the DynamoDb stream?

I would highly recommend you use a lambda to forward your data to Kinesis which would give you all these capabilities.

However if really need to stick with "only" DynamoDb streams.. then depending on the number of consumers, you will need to have additional code that can handle the throttling errors gracefully or/and reduce the dynamodb stream polling period.

onlyplo
  • 86
  • 2