Flink ProcessFunction onTimer first event LONG.MIN_VALUE

Question

I am having a use case: stepA -> stepB -> stepC ... Now I need to use Flink processfunction to monitor the track. for example, once stepA comes in, I set up a timer 10s after A, and when 10s has passed, that timer is triggered, and we check our state to see if stepB has showed up. But here is the problem I have: I am using event-time with AssignerWithPunctuatedWatermarks. But when I print the context.timeservice().currentwatermark() in stepA, it shows LONG.MIN_VALUE. And the watermark of stepB is A's timestamp. I know this is due to

  public long extractTimestamp(
      MyEvent event, long previousElementTimestamp) {
      return event.getTimeStamp();
  }
  
  public Watermark checkAndGetNextWatermark(
      MyEvent event, long extractedTimestamp) {
    return new Watermark(extractedTimestamp);
  }

But what else can I do? Thanks

score 0 · Answer 1 · answered Aug 07 '20 at 10:08

The way that Flink applies watermarking is that watermarks follow the events that were used as evidence for creating the watermark. Thus the current watermark will be LONG.MIN_VALUE during the processing of the first event, after which the watermark will be advanced.

From what you've shared, it isn't clear why this is a problem for your application.

By the way, your watermark generator is assuming that your events are strictly in order, by timestamp. Is that actually true?

Flink ProcessFunction onTimer first event LONG.MIN_VALUE

1 Answers1