0

I’ve been trying to emulate the behavior of a dynamic window, as Flink does not support dynamic window sizes.

My operator inherits from KeyedProcessFunction, and I’m only using KeyedStates to manipulate the window_size. I’m clearing the KeyedStates when my bucket(window) is complete, to reset the bucket size.

My concern is, as Flink does not support dynamic windows, is this approach going against Flink Architecture? Like will it break checkpointing mechanism in distributed systems? It's been noted that I’m only using KeyedStates for maintaining or implementing the dynamic window.

Salekeen Nayeem
  • 1
  • 1

1 Answers1

0

Have you looked into Session Windows?

The session windows assigner groups elements by sessions of activity. Session windows do not overlap and do not have a fixed start and end time, in contrast to tumbling windows and sliding windows. Instead a session window closes when it does not receive elements for a certain period of time, i.e., when a gap of inactivity occurred.

https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/windows/#session-windows

Sean L
  • 48
  • 4