Questions tagged [flink-sql]

Apache Flink features two relational APIs, SQL and Table API, as unified APIs for stream and batch processing.

Apache Flink features two relational APIs:

  1. SQL (via Apache Calcite)
  2. Table API, a language-integrated query (LINQ) interface

Both APIs are unified APIs for stream and batch processing. This means that the a query returns the same result regardless whether it is applied on a static data set or a data stream. SQL queries are parsed and optimized by Apache Calcite (Table API queries are optimized by Calcite).

Both APIs are tightly integrated with Flink's DataStream and DataSet APIs.

667 questions
0
votes
1 answer

Missing artifact org.apache.flink:flink-table:jar:1.10.1

I am trying to add Flink Table dependency in my POM.xml file and the following is the dependency. org.apache.flink flink-table 1.10.1 This throws the…
Ajay Chinni
  • 780
  • 1
  • 6
  • 24
0
votes
1 answer

Flink hadoop implementation problem - Could not find a file system implementation for scheme 'hdfs'

I'm struggling with integration hdfs to flink. Scala binary version: 2.12, Flink (cluster) version: 1.10.1 here is HADOOP_CONF_DIR; and configuration of hdfs is here; This configuration and HADOOP_CONF_DIR also the same in the taskmanager as…
0
votes
1 answer

I got an error for flink k8s ha. job 00000000000000000000000000000000 is not in state RUNNING but SCHEDULED instead. Aborting checkpoint

When I apply flink job to k8s zookeeper ha, I get below error. Our structure is job cluster. 1 job and 1 task. We want to implement while we delete job pod the task still can continue work. job 00000000000000000000000000000000 is not in state…
Jeff
  • 117
  • 10
0
votes
1 answer

Flink streaming table using kafka source and using flink sql to query

I'm trying to read data from kafka topic into DataStream and register DataStream, after that use TableEnvironment.sqlQuery("SQL") to query the data, when TableEnvironment.execute() there is no error and no output. public static void main(String[]…
0
votes
1 answer

Flink SQL : UDTF passes Row type parameters

CREATE TABLE user_log ( data ROW(id String,user_id String,class_id String) ) WITH ( 'connector.type' = 'kafka', ... ); INSERT INTO sink SELECT * FROM user_log as tab, LATERAL TABLE(splitUdtf(tab.data)) AS T(a,b,c); UDTF Code: public…
Zheng
  • 21
  • 2
0
votes
1 answer

How to add new rows to an Apache Flink Table

Is it possible to add a new record/row to a flink table? For example i have the following table configuration: ExecutionEnvironment env = TableEnvironmentLoader.getExecutionEnvironment(); BatchTableEnvironment tableEnv =…
0
votes
1 answer

How to update the Broadcast state in KeyedBroadcastProcessFunction in flink?

I am new to Flink i am doing a pattern matching using apache flink where the list of patterns are present in broadcast state and iterating through the patterns in processElements function to find the pattern matched and i am reading this patterns…
YRK
  • 153
  • 1
  • 1
  • 22
0
votes
0 answers

Flink SQL nested elements Avro

I have Avro schema that contains nested structure and when querying using Flink SQL, we are getting below error. Exception in thread "main" java.lang.AssertionError at org.apache.calcite.sql.parser.SqlParserPos.sum_(SqlParserPos.java:236) …
user1261215
0
votes
1 answer

Create pageable JDBC source for Flink Job

For processing data from DB I am using flink. I have created input with jdbc. val inputFormat = JDBCInputFormat.buildJDBCInputFormat() .setDrivername(driver) .setDBUrl(url) .setUsername(username) …
Uladzislau Kaminski
  • 2,113
  • 2
  • 14
  • 33
0
votes
2 answers

Simple TableAPI SQL query doesn't work on Flink 1.10 and Blink

I want to define Kafka connector using TableAPI and run SQL over such described table (backed by Kafka). Unfortunately, it seems that Rowtime definition doesn't work as expected. Here's a reproducible example: object DefineSource extends App { …
bottaio
  • 4,963
  • 3
  • 19
  • 43
0
votes
1 answer

Apache Flink Table query result as string values

I am writing a query from a flink table api to retrieve a record. Then check if a record was found and if so, get the string value of each of the record's column values. i.e. users: |id | name | phone | |---|------|-------| | 01| sam | 23354 | |…
0
votes
1 answer

PARTITION BY with TUMBLE causes exception in FlinkSQL

I would like to select last element of processing window using FlinkSQL. Tried to achieve that with ROW_NUMBER in Blink planner. Tried following query: SELECT * FROM ( SELECT key, value, ROW_NUMBER() OVER w AS rn FROM InputTable WINDOW w AS…
bottaio
  • 4,963
  • 3
  • 19
  • 43
0
votes
0 answers

LAST_VALUE over window in FlinkSQL

I would like to run following query SELECT key, LAST_VALUE(value) OVER (PARTITION BY key ORDER BY ts) AS value FROM [table] GROUP BY key, TUMBLE(ts, INTERVAL '5' MINUTE) I would expect that LAST_VALUE would return last value of each time window.…
bottaio
  • 4,963
  • 3
  • 19
  • 43
0
votes
2 answers

FlinkSQL - select last

I would like to emit last record of a time window. This can easily be done with maxBy in regular Flink but I cannot get it to work through SQL API. What I want is: SELECT LAST(attribute) FROM [table] GROUP BY key, TUMBLE(ts, INTERVAL '1'…
bottaio
  • 4,963
  • 3
  • 19
  • 43
0
votes
1 answer

how can I get job submitting time and use it in Flink application?

I'm currently developing a stream processing application, one of the functionality is to take events that happen in the time zone [time of submitting the job, time of submitting the job + T ]. how can access to that particular variable (time of…