0

I have a table created as delta table. I deleted the delta file from the azure container but I have the table on databricks database. How can I retrieve the data.

I tried reading the file from database but it gives me error as: spark.sql("select * from employee.information") "AnalysisException: Delta table "employee.information" doesn't exist".

Siddhu
  • 19
  • 4
  • `I deleted the delta file from the azure container but I have the table on databricks database`. What do you mean by this? – Saideep Arikontham May 09 '23 at 06:59
  • So I created a delta table as: CREATE TABLE DB.TABLENAME IF NOT EXIST () USING DELTA LOCATION /mnt/subscription/dev/container/xyz. So I deleted the delta file from xyz location but the data still exist in DB.TABLENAME – Siddhu May 09 '23 at 09:23
  • So you deleted the delta file before creating the table in databricks? – Saideep Arikontham May 09 '23 at 09:24
  • Actually, I created the table and delta file at once. And now I just wanted to drop the table but accidently I deleted the delta files and now I want to check if we can create the delta files again with the data that exist in my table. – Siddhu May 09 '23 at 09:34

1 Answers1

0
  • Since you want to create the table again with delta log files, you can use the previous files from the folder in your container. Without the delta log files, this would be a folder with parquet files. You can read it directly in the following way.
  • This is an image of the contents in my delta table inside my container with delta long files.

enter image description here

  • When I delete the delta log files, only the parquet files remain. So, using the following code, you can read this data directly without creating any table:
df1 = spark.read.format('parquet').option("header",True).load('/mnt/repro/my_data/*.parquet')
df1.show()

enter image description here

  • Now write this dataframe, overwriting the previous folder to get back the data (table) with delta log files as well.
df1.write.mode("overwrite").format('delta').option("header",True).save('/mnt/repro/my_data')
Saideep Arikontham
  • 5,558
  • 2
  • 3
  • 11