0

I am trying to flatten an array using UNNEST function in the Table API.

Am I doing something wrong or it is not a supported function ? This page suggests it though: https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html

Thanks !

Code

Python udf

@udf(input_types=DataTypes.STRING(), result_type=DataTypes.ARRAY(DataTypes.STRING()))
def explode(s):
    return 3*[s]

t_env.register_function("explode", explode)

Processing

tab = t_env.from_path('mySource').select("id, explode(dummy) as dummy_list")
t_env.register_table("temp_table", tab)
t_env.sql_query("SELECT t.item as dummy_item FROM UNNEST(select dummy_list from temp_table) AS t(item)").insert_into("mySink")

Execution

t_env.execute("dummy_unnest")

Error

TableException: Cannot generate a valid execution plan for the given query: 

LogicalProject(dummy_item=[$0])
  Uncollect
    LogicalProject(EXPR$0=[org$apache$flink$table$functions$python$PythonScalarFunction$908596b4671476ee325743dba92ed6c7($1)])
      LogicalTableScan(table=[[default_catalog, default_database, mySource]])

This exception indicates that the query uses an unsupported SQL feature.
Please check the documentation for the set of currently supported SQL features.
py-r
  • 419
  • 5
  • 15

1 Answers1

0

I think you can change the query to

select id, dummy_item from temp_table CROSS JOIN UNNEST(dummy_list) AS t (dummy_item)
Xingbo Huang
  • 363
  • 1
  • 5
  • :) In the future, if you have questions with pyflink, you can also ask directly in the flink user mail list to get a more timely answer. – Xingbo Huang Nov 04 '20 at 01:26
  • Thanks I will. I thought this kind of question might benefit to a wider public. – py-r Nov 04 '20 at 06:31