Flink Timestamp monotony violated

Asked Nov 02 '16 at 12:45

Active Nov 02 '16 at 12:45

Viewed 1,112 times

I'm consuming data using a Kafka Consumer 0.9 and after some initial filtering for valid events and mapping to POJO I am setting AscendingTimestampExtractor.

using Joda Time library I'm returning the milliseconds since epoch as follows: return DateTime.parse(evnt.timestamp).getMillis();

This seems to result in WARN org.apache.flink.streaming.api.functions.timestamps.AscendingTimestampExtractor - Timestamp monotony violated: 1478048406982 < 1478051502295

Of course the ordering is different but what I don't understand is how make sure these are in order? Should I be ordering these based on this timestamp and then performing any keyBy or timeWindow?

I haven't really found much information on this so any help would be appreciated.

asked Nov 02 '16 at 12:45

AtharvaI

1,160
3
16
27

1

What does `evnt.timestamp` look like? If you're going through time zone conversion, that could well be the issue. (Those timestamps look like they are indeed just either side of a DST transition.) – Jon Skeet Nov 02 '16 at 12:47
2

Are the elements in your Kafka partition monotonic wrt `evnt.timestamp`? – Till Rohrmann Nov 02 '16 at 13:11
`evnt.timestamp` is ISO 8601 formatted in UTC. example: `2016-11-02T01:51:11.363Z` – AtharvaI Nov 02 '16 at 13:18
I meet same problem. Do you solved it?@AtharvaI – Libraco Mar 04 '20 at 08:01
I did not. I haven't used Flink since I posted the question. Not able to help. – AtharvaI Jun 19 '20 at 14:05

Flink Timestamp monotony violated

0 Answers0