Error connecting to DataLake(ADLS Gen2) store from databricks

Question

I am trying to connect to dataLake Gen2 storage from databricks python, unfortunately I am running into error.

Code:

dbutils.fs.ls("abfss://<fsystem name>@<storage name>.dfs.core.windows.net/<folder name>")

Error Message:

Configuration property .dfs.core.windows.net not found.

I doubt if it is something to do with my mount code? Additionally I have added Tenant ID to container "manage access" using storage explorer.

here is my mount code:

configs = {"fs.azure.account.auth.type": "OAuth",
       "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
       "fs.azure.account.oauth2.client.id": "<client ID>",
       "fs.azure.account.oauth2.client.secret":  "secret",
       "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/directory id/oauth2/token"}
dbutils.fs.mount(  source = "abfss://filesystem name@<storage name>.dfs.core.windows.net/",  mount_point = /mnt/soldel",  extra_configs = configs)

Mount code ran fine, with no errors. Please suggest

score 3 · Accepted Answer · answered Oct 04 '19 at 09:50

Note: You cannot access the Azure Data Lake Gen2 account without configuring the storage account with Databricks.

This is expected error message because you haven't configured storage account with databricks to list filesystem.

Kindly check out the error message and see the correct process of list filesystem in Databricks.

For more details, refer "Databricks - Azure Data Lake Storage Gen2".

Hope this helps.

Error connecting to DataLake(ADLS Gen2) store from databricks

1 Answers1