2

Lets say I have window2 -> window1 (window1 goes before window2). Lets say offsets are: (start2, end2) and (start1, end1) correspondingly.

Since each window processing might take different time, window2 might finish processing before window1. Then:

1) window2 process commits end2 offset to Kafka

2) window1 processing fails

What will happen then? Will spark streaming skip data between start1 and start2? How does it work?

0 Answers0