Databricks error while trying to create delta table on ADLS Gen2

Question

I am working with Azure Databricks to create a delta table on Azure Data Lake Storage Gen2 and running into error.

Code:

dataframe.write.format("delta").mode("overwrite").option("path","abfss://<ContainerName>@<StorageAccount>.dfs.core.windows.net").saveAsTable("test_table")

Error:

IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: abfss://<ContainerName>@<StorageAccount>.dfs.core.windows.net_delta_log

do you really have path as `abfss://@.dfs.core.windows.net` or it was just changed to hide names? — Alex Ott, Jul 22 '21 at 06:51

score 1 · Accepted Answer · answered Jul 22 '21 at 06:54

1

You need to set path to the specific directory inside your Data Lake, not onto the top of the container. Something like:

ContainerName = "container"
StorageAccount = "account"
table_path = f"abfss://{ContainerName}@{StorageAccount}.dfs.core.windows.net/test-table"
dataframe.write.format("delta").mode("overwrite")\
  .option("path", table_path).saveAsTable("test_table")

answered Jul 22 '21 at 06:54

Alex Ott

80,552
8
87
132

Hi @AlexOtt, I am running into this below error Py4JJavaError: An error occurred while calling saveAsTable Failure to initialize configuration. – paone Jul 22 '21 at 22:34
As I said, try to specify directory name – Alex Ott Jul 23 '21 at 03:55
don't we need to provide SAS token? – Vinay G Jun 23 '23 at 07:09
there are different authentication methods, but yes - you need to have correct configuration – Alex Ott Jun 23 '23 at 07:17

Databricks error while trying to create delta table on ADLS Gen2

1 Answers1

Linked