0

I have a java consumer (KCL implementation) to consume messages from 1 Kinesis stream with 2 shards.

The KCL creates 2 RecordProcessors correlates to each shard.

Then I put my machine to sleep, so JVM suspended. When wake up, the KCL library creates 2 extra RecordProcessors in addition to the original 2 RecordProcessors. So for each shard, there are 2 processors which causes confliction.

The expected behavior is not to spawn 2 new RecordProcessors while the original RecordProcessors are still in place. Where is the code in KCL that deals with the logic of spawning new RecordProcessors?

ethan
  • 1,881
  • 2
  • 17
  • 31
  • The [KCL library is open-source](https://github.com/awslabs/amazon-kinesis-client). However, I wouldn't focus on the logic that creates a RecordProcessor. You can create any number of RecordProcessors, but they cannot have any effect on processing unless they can successfully take lease ownership of a shard. In order to take lease ownership, the original processor should lose its lease, and essentially shut down, leaving you with two new processors and everything running smoothly. Can you provide more details on what kind of conflict / issue are you seeing? I can't reproduce it... – Krease Jun 11 '18 at 17:29
  • Here is the issue that I see. Originally shard and record processor is one-to-one (2x shards and 2x record processors). After system sleep and wake up, two extra record processors were started, while the original 2 were still running. So shard:RP ratio is 1:2, essentially 2 record processors are process the same shard at the same time, they get the lease in turn, causing lots of duplicate records be reprocessed. – ethan Jun 12 '18 at 16:02
  • You can have 100 record processors and 2 shards - the number of processors is not the issue - what would happen in that case is that 98 of them "sleep" because there are only 2 leases (one lease per shard). Examine what's going on with lease ownership that's causing multiple processors to own a single lease. – Krease Jun 12 '18 at 16:57
  • Is the lease ownership managed by the dynamodb table? In my case, the app is actually deployed inside tomcat, it's one instance that has multiple record processors, apparently all the record processors are active. From the dynamodb table, the lease owner is set to the appname plus ip, it remains the same. Lease switch is 0. – ethan Jun 13 '18 at 18:46
  • yes, lease ownership is managed by the dynamodb table. – Krease Jun 13 '18 at 18:57

0 Answers0