2

I am trying to control number of records fetched in processRecords method of KCL, here is my KCL configuration

new KinesisClientLibConfiguration(
        config.getApplicationName(), config.getStreamName(), awsCredentialsProvider, workerId)
        .withInitialPositionInStream(InitialPositionInStream.LATEST)
        .withRegionName(config.getRegion())
        .withMaxRecords(config.getMaxRecords())

My config.getMaxRecords() is set to 500 but i am still getting ~1000-1500 records in processRecords method, i want strict upper limit as 500 since my downstream process can not handle more than that.

Amit Kumar
  • 825
  • 1
  • 9
  • 19
  • What I see here looks right - I hate to ask, but are you sure the config value is actually 500, and that this is the config being used? It wouldn't be the first time questions like this came from typos – Krease Apr 08 '18 at 18:39
  • 1
    https://github.com/awslabs/amazon-kinesis-client/issues/285 – ethan Jun 01 '18 at 04:38
  • I have the same issue with KCL 1.9.1, I set max records as 300-500, but still getting around 1000 during the peak time. – WineYe Jul 01 '19 at 02:13

0 Answers0