1

I'm trying to figure out the poll records mechanism for Kafka over SCS in a K8s environment.

  1. What is the recommended way to control max.poll.records?

  2. How can I poll the defined value?

  3. Is it possible to define it once for all channels and then override for a specific channel? (referring to this comment form documentation):

To avoid repetition, Spring Cloud Stream supports setting values for all channels, in the format of spring.cloud.stream.kafka.default.consumer.=. The following properties are available for Kafka consumers only and must be prefixed with spring.cloud.stream.kafka.bindings..consumer..")

  1. Is this path supported: spring.cloud.stream.binding.<channel name>.consumer.configuration?

  2. Is this: spring.cloud.stream.**kafka**.binding.<channel name>.consumer.configuration?

  3. How are conflicts being resolved? Let's say in a case where both spring.cloud.stream.binding... and spring.cloud.stream.**kafka**.binding... are set?

I've tried all mentioned configurations, but couldn't see in the log what is the actual poll.records and frankly the documentation is not entirely clear on the subject.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Itamar Pasalski
  • 11
  • 3
  • _couldn't see in the log what is the actual poll.records_ - This sounds like a logging problem since, by default, `ConsumerConfig` output does show this value – OneCricketeer Feb 13 '23 at 17:39

1 Answers1

0

These are the configurations:

  1. spring.cloud.stream.kafka.default.consumer.configuration.max.poll.records - default if nothing else specified for given channel

  2. spring.cloud.stream.kafka.bindings..consumer.configuration.max.poll.records

Itamar Pasalski
  • 11
  • 3
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Feb 23 '23 at 21:58