0

was wondering if somebody could clarify the difference between these two metrics when used in an enhanced fan-out (KCL 2.x) application.

I believe the KCL 'MillisBehindLatest' measure the iterator age used in the GetRecords call. However, when using KCL 2.x the records are pushed to consumer over HTTP/2 instead, and should use the SubscribeToShardEvent.MillisBehindLatest instead

In practice, our KCL 2.x application emits both of these metrics to cloudwatch. It is unclear why MillisBehindLatest are being emitted if the polling with iterator is not occuring.

John Doe
  • 205
  • 6
  • 17

1 Answers1

0

The difference between these two metrics is that the SubscribeToShardEvent.MillisBehindLatest metric defines how far behind consumer is across all shards, i.e. maximum delay across all shards, meanwhile MillisBehindLatest metrics are shard-level specific, i.e. delay for each specific shard.

For example, in case of 3 shards, you can have MillisBehindLatest for shardId-000000000000 equal to 0, for shardId-000000000001 equal to 120 and for shardId-000000000002 equal to 60, then the value of SubscribeToShardEvent.MillisBehindLatest will be 120.

Mikalai Lushchytski
  • 1,563
  • 1
  • 9
  • 18