1

I am using Kafka stream to perform aggregation to find windowed average. ( 3 seconds window size)

I tried both tumbling window and hoping window , but the output topic have results only in every 30 seconds, not in every 3 seconds.

I changes the window size to higher values(1 min) and lower values (1 sec). For all, the output generation rate is close to 25 to 30 seconds. The internal window size is correct.

Is there any configuration in Kafka that increase the output generation speed.

Jobs
  • 1,257
  • 2
  • 14
  • 27

1 Answers1

1

The Kafka streams will generate output ~30 seconds because the stream configuration StreamsConfig.COMMIT_INTERVAL_MS_CONFIG is 30000ms by default.

I believe COMMIT_INTERVAL_MS_CONFIG also decides the time taken to finish kafka-streams task and publish it to destination topic.

Please refer to the configurations mentioned in below Kafka documentation:

https://kafka.apache.org/25/documentation/streams/developer-guide/config-streams.html

Akhil K Kamal
  • 100
  • 7