Stack Overflow
Stack Overflow

Questions tagged [apache-iceberg]

Apache Iceberg is a high-performance table format to enable analytics purposes. It allows SQL tables to be consumed by analytics tools such as Apache Spark, Apache Flink, Apache Hive, Trino, PrestoDB, Impala, StarRocks, Doris, and Pig.

Apache Iceberg (often referred only as Iceberg) is a high-performance table format to enable analytics purposes. It allows SQL tables to be consumed by analytics tools such as Apache Spark, Apache Flink, Apache Hive, Trino, PrestoDB, Impala, StarRocks, Doris, and Pig.

68 questions
0
votes
0 answers

Apache iceberg: how to set write.metadata.previous-versions-max

Having many historical metadata files in apache iceberg helps us to produce a linear history of table versions and ensures that concurrent writes are not lost. In Apache iceberg there is a table write property…
wbrycki
  • 121
  • 1
  • 8
0
votes
1 answer

Apache Fink & Iceberg: Not able to process hundred of RowData types

I have a Flink application that reads arbitrary AVRO data, maps it to RowData and uses several FlinkSink instances to write data into ICEBERG tables. By arbitrary data I mean that I have 100 types of AVRO messages, all of them with a common…
nach0
  • 379
  • 1
  • 3
  • 14
0
votes
1 answer

how to use replaceWhere option with Apache iceberg while writing data

I'm currently trying to write data using Iceberg to an external Hive table which is partitioned by partition_date column. Before writing the data with Iceberg format, test table has 2 rows, ("2015-01-02", "S01233",…
Leroy Mikenzi
  • 792
  • 6
  • 22
  • 46
0
votes
0 answers

spark2 cbo on iceberg table

I am using Spark2.4 with an iceberg table. I want to enable CBO but I do not find a way to calc table stats. The table is created using Iceberg Catalog API and data is populated using Spark DataFrame. Is there a way to do that?
igreenfield
  • 1,618
  • 19
  • 36
0
votes
1 answer

java.lang.IncompatibleClassChangeError: class org.apache.spark.sql.catalyst.plans.logical.DynamicFileFilterWithCardinalityCheck has interface

While launching the spark-shell with iceberg dependencies, we got the following error: spark-shell \ --packages org.apache.iceberg:iceberg-spark3-runtime:0.13.0 \ --conf…
Ranga Reddy
  • 2,936
  • 4
  • 29
  • 41
0
votes
1 answer

"Iceberg query cannot be parsed" when trying to create Iceberg table with MAP column data type in Athena?

According to the Athena Iceberg documentation, the map type is supported. Why do neither of these statements work? CREATE TABLE iceberg_test1 (id string, themap map) LOCATION 's3://mybucket/test/iceberg1' TBLPROPERTIES ( 'table_type' = 'ICEBERG'…
Alex R
  • 11,364
  • 15
  • 100
  • 180
0
votes
1 answer

Is there a way to inspect all queries that are submitted to apache iceberg?

I am new to Apache iceberg. I would like to know if there is a way I can inspect the queries that are submitted to iceberg. Is there a way to do that?
hba
  • 7,406
  • 10
  • 63
  • 105
-1
votes
0 answers

For data lake storage in AWS S3. What are the advantages of Apache Iceberg over raw parquet Tables?

We are building a data lake and we are storing the data in S3 in parquet format. We are extracting and transforming with Glue. It was proposed that we use Apache Iceberg as table format instead of regular parquet files in partitions. I understand…
Cristobal Sarome
  • 178
  • 11
1 2 3 4
5