If we create a delta table(let's say in Azure ADLS), then the data is stored into ADLS in parquet file format. How are we able to use SQL queries on the stored data? Does Spark convert SQL queries into Java code internally?
Asked
Active
Viewed 34 times
1 Answers
0
SQL isn't supported by Delta directly, but it's a part of some engine, like, Apache Spark that translates SQL commands into the corresponding execution plan and executes it. Spark supports many other file formats - CSV, JSON, Delta, Parquet, ..., and mostly works the same, but some file formats, like, Delta can implement additional optimizations, like, avoiding reading of not necessary data files, etc.

Alex Ott
- 80,552
- 8
- 87
- 132