Creating a Tumbling Window for a Flink table

Question

I'm trying to create a tubmbling window for a Flink table in Java and query data for that window

Table table = tEnv.sqlQuery(query.getQuery());
// create a new column 'EventTime' of type Timestamp from 'EventTimetamp' which is a string
table = table.addColumns($("EventTimestamp").toTimestamp().as("EventTime"));
WindowGroupedTable windowedTable = table.window(Tumble.over("10.minutes").on($("EventTime").proctime())
    .as("w"))
    .groupBy($("w"), $("GroupingColumn"));
table = windowedTable.select($("*"));

However, I'm getting this error Expected LocalReferenceExpression. Got: EventTime when it tries to execute windowedTable.select($("*")). Does anyone know what this error means and how to solve it ?

Kenank · Answer 1 · 2022-12-07T14:24:28.137

0

The problem is with .addColumns($("EventTimestamp") . addCoumns() takes an expression as input. Take a look at the example from docs.

Table orders = tableEnv.from("Orders");
Table result = orders.addColumns(concat($("c"), "sunny"));

In this example, the column "c" already exists and you tell flink to concatane the value in column "c" with string "sunny" and add the new value as a new column.

So the meaning of your code

table.addColumns($("EventTimestamp")

in English is "select EventTimeStamp from table and add the values as a new column" As your table does not have a column "EventTimestamp" yet, expression $("EventTimestamp") returns error.

EDIT: The problem might be related about ROWTIME. Please take a look at my answer using-tumble-failing-on-timestamp

        Table tableCpuDataCalculatedTemp = tableEnv.fromDataStream(streamCPUData, Schema.newBuilder()
                        .column("deviceId", DataTypes.STRING())
                        .column("cpuUsageAvarage", DataTypes.DOUBLE())
                        .column("cpuLoad15Average", DataTypes.DOUBLE())
                        .column("recordCount", DataTypes.INT())
                        .column("time_Insert_ts", DataTypes.TIMESTAMP(3))
                        .watermark("time_Insert_ts", "time_Insert_ts")
                        .build());

edited Dec 07 '22 at 14:24

answered Dec 01 '22 at 08:33

Kenank

321
1
10

I have an existing column named `EventTimestamp` in my table, and the issue is not with adding a new column, I have confirmed by printing the table schema that the new column i.e. `EventTime` is added to the table. The problem is that after creating the tumbling window, when I call `windowedTable.select($("*"));` it throws this exception `Expected LocalReferenceExpression. Got: EventTime` – shepherd Dec 01 '22 at 17:30
Can you put the schema here. – Kenank Dec 01 '22 at 20:00
A column should be marked as ROWTIME. I `ve edited my original answer and added a link. Please check. – Kenank Dec 01 '22 at 20:52
Do you know how to do this directly on an existing Flink Table object instead of creating a new Table (or in a datastream)? – shepherd Dec 05 '22 at 17:54
please take a look at .watermark() method. It can be applied on an existing table I think. I editted my original answer too. – Kenank Dec 07 '22 at 14:23

Creating a Tumbling Window for a Flink table

1 Answers1