0

I have created my very first Delta table using a Notebook in Azure Synapse. I am now trying to read it but I am getting an error. Here is the code I have written (I have masked some of the information) :

df = spark.read.format("delta").load('https://xxx.blob.core.windows.net/footballdata/delta/football_data/part-00000-c9b34d19-feb5-4774-8e05-0f5d6ee89e47-c000.snappy.parquet')

I get the following error:

Py4JJavaError                             Traceback (most recent call last)
/tmp/ipykernel_7339/2373797746.py in <module>
----> 1 df = spark.read.format("delta").load('https://xxx.blob.core.windows.net/footballdata/delta/football_data/part-00000-c9b34d19-feb5-4774-8e05-0f5d6ee89e47-c000.snappy.parquet')

Just a note, the delta_log file is located in the football_data folder.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
HamidBee
  • 187
  • 1
  • 7
  • 2
    You don't need to include the path to the specific parquet file you're reading in `spark.read.format('delta').load()`. Just `https://xxx.blob.core.windows.net/footballdata/delta/football_data` as the path for the delta table is enough. – ARCrow Aug 13 '23 at 05:12
  • 1
    Please put it down as an answer so I can mark it as a solution. – HamidBee Aug 13 '23 at 14:54

1 Answers1

1

You don't need to include the path to the specific parquet file you're reading in spark.read.format('delta').load(). Just https://xxx.blob.core.windows.net/footballdata/delta/football_data as the path for the delta table is enough.

ARCrow
  • 1,360
  • 1
  • 10
  • 26