Stack Overflow
Stack Overflow

Questions tagged [flink-sql]

Apache Flink features two relational APIs, SQL and Table API, as unified APIs for stream and batch processing.

Apache Flink features two relational APIs:

  1. SQL (via Apache Calcite)
  2. Table API, a language-integrated query (LINQ) interface

Both APIs are unified APIs for stream and batch processing. This means that the a query returns the same result regardless whether it is applied on a static data set or a data stream. SQL queries are parsed and optimized by Apache Calcite (Table API queries are optimized by Calcite).

Both APIs are tightly integrated with Flink's DataStream and DataSet APIs.

667 questions
0
votes
0 answers

Can't get results from flink SQL query

I'm facing a problem in which I don't get results from my query in Flink-SQL. I have some informations stored in two Kafka Topics, I want to store them in two tables and perform a join between them in a streaming way. These are my flink instructions…
Gatsby
  • 365
  • 1
  • 5
  • 17
0
votes
1 answer

Flink - Dataset - Can Flink respect the order of processing on multiple flows / input ?

In my Flink batch program (DataSet / Table ), I am reading multiple file, this is producing differents flows, do some processing, and save it with output format As flink is using dataflow model, and my flows are not really related, it is processing…
Eldinea
  • 165
  • 1
  • 14
0
votes
1 answer

Issue with Batch Table API in Flink 1.5 - Complains of need of Streaming API

I'm trying to create a Batch oriented Flink job, with Flink 1.5.0 and wish to use the Table and SQL APIs to process the data. My problem is trying to create the BatchTableEnviroment I get a compiling error BatchJob.java:[46,73] cannot access…
olibs
  • 99
  • 9
0
votes
1 answer

Does Flink SQL support to run projections in parallel

I am running a SQL with multiple projections each are time consuming, for example: SELECT UDF1(id), UDF2(id) FROM prod The UDF1 and UDF2 may be time consuming function, but it looks like Flink SQL runs UDF1 and UDF2 sequentially, my question is…
yinhua
  • 337
  • 4
  • 18
0
votes
1 answer

Submit Flink Job on Yarn cluster to restores its state from it's last savepoint

I am running these Flink job in a Yarn cluster. When the flink job fails or I restart it I want the Flink job to use the savepoint that was created before it restarted. How do can restart the Flink job with this savepoint. I am using Flink 1.4.2.
user3107673
  • 423
  • 4
  • 9
0
votes
1 answer

Joining two streams and managing state in Flink

I have two nifi streaming source in flink, I need to perform joins on this two sources. Which will be the better solution to do that? Is it joins api provided by DataStreams or table…
Rilwan
  • 88
  • 11
0
votes
0 answers

Terminate request to database in flink operation

I am trying to work with Flink and Cassandra. Both are massively parallel environments, but I have difficulties to make them working together. Right now I need to make an operation for parallel read from Cassandra by different token ranges with the…
Sergey Okatov
  • 1,270
  • 16
  • 19
0
votes
1 answer

Flink Read CSV across multiple host

I have a cluster like https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/cluster_setup.html and in which worker I have multiple CSV corresponding to a shard for each host. I want to use the table API to calculate a sum of a CSV…
dfvt
  • 87
  • 2
  • 12
0
votes
2 answers

Read Data From Redis Using Flink

I am completely new to Flink. May this question is repeated but found only one link and that is not understandable for me. https://stackoverflow.com/a/44294980/6904987 I stored Data in Redis in Key Value format example Key is UserId and UserInfo is…
Darshan Manek
  • 155
  • 1
  • 2
  • 12
0
votes
1 answer

How to write a SQL for a calculation based on incremental window of batch table

My requirement is to calculate based on an incremental size window for a batch table. For example, the first window has 1 row, the second window has 2 rows(including 1 row from the 1st window and a new row), then 3 rows in the 3rd window(including 2…
yinhua
  • 337
  • 4
  • 18
0
votes
1 answer

Flink Stream SQL order by

I have a streaming input, say stock price data (including multiple stocks), and I want to do a ranking by their price every 1 minutes. The ranking is based on all stocks' latest price and needs to sort all of them no matter if it was updated in the…
yinhua
  • 337
  • 4
  • 18
0
votes
0 answers

Set Field Name Tuple TableSource FlinkDB

I have data stored in Tuples like Tuple6 and I make SQL statements like select f1 from test. My problem is I want to set column names instead of f1, f2, f3. Can I still have Tuple and set column Names? Some code Input format public Tuple6
dfvt
  • 87
  • 2
  • 12
0
votes
2 answers

Apache Flink: Unable to convert the Table object to DataSet object

I am using the Table API on Flink 1.4.0. I have some Table objects to be convert to a DataSet of type Row. The project was built using Maven and imported on IntelliJ. I have the following code and the IDE cannot resolve the method…
Ranjan
  • 91
  • 8
0
votes
1 answer

Use of Maps in Flink SQL

If you register a table and one of its fields is a map (extra in this case“) the following statement works just fine: SELECT f1, f2 FROM customers WHERE extra['sportPrefs'] = 'Football'; Now try to reference a key that does not exist in the…
Slava Minyailov
  • 11
  • 1
0
votes
1 answer

How to attach schema to a Flink DataStream - on the fly?

I am dealing with a stream of database mutations, i.e., a change log stream. I want to able to transform the values using a SQL query. I am having difficulty putting together the following three concepts RowTypeInfo, Row, and DataStream. NOTE: I…
user758988
  • 67
  • 1
  • 7
1 2 3
…
44
45