Stack Overflow
Stack Overflow

Questions tagged [flink-sql]

Apache Flink features two relational APIs, SQL and Table API, as unified APIs for stream and batch processing.

Apache Flink features two relational APIs:

  1. SQL (via Apache Calcite)
  2. Table API, a language-integrated query (LINQ) interface

Both APIs are unified APIs for stream and batch processing. This means that the a query returns the same result regardless whether it is applied on a static data set or a data stream. SQL queries are parsed and optimized by Apache Calcite (Table API queries are optimized by Calcite).

Both APIs are tightly integrated with Flink's DataStream and DataSet APIs.

667 questions
2
votes
2 answers

Flink SQL on real time data and historical data

I am working on an application where I want to run Flink SQL on real time events and past events.I tried a POC where Flink runs SQL on streaming sources such as Kafka, SQL query only returns new events / changes. But, I want to run SQL on whole…
user1261215
2
votes
1 answer

Flink SQL : Use changelog stream to update rows in Dynamic Table

I have a stream that contains JSON messages that look like this : {"operation":"CREATE","data":{"id":"id-1", "value":"value-1"}} {"operation":"CREATE","data":{"id":"id-2",…
Nakeuh
  • 1,757
  • 3
  • 26
  • 65
2
votes
1 answer

Using BlinkPlanner in Flink

I was reading the documentation about how to use the "BlinkPlanner" and I found that it is possible to create an instance of TableEnvironmentImpl using the code above: import org.apache.flink.table.api.{EnvironmentSettings, TableEnvironment} val…
Salvatore Rapisarda
  • 97
  • 6
2
votes
1 answer

Flink : Rowtime attributes must not be in the input rows of a regular join

Using flink SQL API, I want to join multiple tables together and do some computation over time window. I have 3 table coming from CSV files, and one coming from Kafka. In the Kafka table, I have a field timestampMs, that I want to use for my time…
Nakeuh
  • 1,757
  • 3
  • 26
  • 65
2
votes
1 answer

How to add uid to operator in Flink Table API?

As is highly recommended by the documentation, I want to add uids to my operators in Flink for the purpose of savepointing. My job uses the Table API. I have not found in the documentation how to add uids to operators with a SQL query. My code looks…
Stevenyc091
  • 195
  • 1
  • 2
  • 22
2
votes
1 answer

Using ROW() for nested data structure

I've been successfully using JsonRowSerializationSchema from the flink-json artifact to create a TableSink and output json from SQL using ROW. It works great for emitting flat data: INSERT INTO outputTable SELECT ROW(col1, col1) FROM…
BenoitParis
  • 3,166
  • 4
  • 29
  • 56
2
votes
1 answer

Why does Flink SQL use a cardinality estimate of 100 rows for all tables?

I wasn't sure why the logical plan wasn't correctly evaluated in this example. I looked more deeply in the Flink base code and I checked that when calcite evaluate/estimate the number of rows for the query in object. For some reason it returns…
Salvatore Rapisarda
  • 97
  • 6
2
votes
1 answer

Flink on yarn use yarn-session or not?

There are two methods to deploy flink applications on yarn. The first one is use yarn-session and all flink applications are deployed in the session. The second method is each flink application deploy on yarn as a yarn application. My question is…
Longxing Wei
  • 171
  • 2
  • 17
2
votes
2 answers

Equivalent to "from_unixtime(bigint unixtime)" in Flink-SQL

I'm searching for the equivalent to the function from_unixtime(bigint unixtime), which exists in Spark-SQL, in Flink-SQL. My aim is to convert this format : 1439799094 into this format : 2015-05-18 05:43:37
Gatsby
  • 365
  • 1
  • 5
  • 17
2
votes
2 answers

Event time window in Flink does not trigger

When I use flink event time window, the window just doesn't trigger. How can I solve the problem, and are there any ways to debug?
Hellen
  • 3,472
  • 5
  • 18
  • 25
2
votes
2 answers

Unable to print a CSV file using Flink Table API

I am trying to read a file with 34 fields in it to be printed on console using Netbeans. However, all I am able to print is the schema. Because the option to print is missing in this particular version of Flink used with csvreader. Please see the…
Merlin Sundar
  • 65
  • 1
  • 2
  • 8
2
votes
1 answer

leftOuterJoin throws TableException: Unsupported join type 'LEFT'

I'm trying to run a left outer join on two tables and convert the results to a DataStream. All the joins I've done before using flink have been inner joins, and I have always followed the join with a .toRetractStream[MyCaseClass](someQueryConfig).…
gseccles
  • 47
  • 1
  • 5
2
votes
2 answers

Flink dynamic scaling 1.5

This is regarding dynamic rescaling in Flink 1.5 I am using Yarn for running Flink jobs. I start these jobs with a static resource. Is there any option to scale out these job by itself in specific conditions like if there's a memory issues. In…
user3107673
  • 423
  • 4
  • 9
2
votes
1 answer

Apache Flink: How to group every n rows with the Table API?

Recently I am trying to use Apache Flink for fast batch processing. I have a table with a column:value and an irrelevant index column Basically I want to calculate the mean and range of every 5 rows of value. Then I am going to calculate the mean…
Jin.J
  • 353
  • 1
  • 4
  • 13
2
votes
1 answer

How to join three or more datastreams /tables on a given key and a common window by datastrem API or Flink Table API/SQL?

I want to Join three or more data streams or tables on a given key and a common window. however I don't know how to correctly write the code. The official document https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/stream/operators/…
YuFeng Shen
  • 1,475
  • 1
  • 17
  • 41
1 2 3
…
44 45