2

I try to use org.apache.hadoop.fs.FileSystem.get(Config...); method to get on azure storages.

In case of Azure Data Lake Gen2 I use URI like:

 abfs://mydlfilesystem@mydlaccount.dfs.core.windows.net/my_path

and set properties:

   fs.defaultFS = "abfs://mydlfilesystem@mydlaccount.dfs.core.windows.net/my_path"
   fs.adl.oauth2.access.token.provider.type = "ClientCredential"
   fs.azure.ssl.channel.mode = Default_JSSE"
   fs.azure.account.oauth.provider.type = "org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider"
   fs.azure.account.auth.type = "OAuth"
   fs.azure.account.oauth2.msi.tenant = "MyTenantId"
   fs.azure.account.oauth2.client.id = "MyClientId"

It does work properly.

But If I try to access Azure Blob Storage. I use URI like:

   wasb://mycont@mybsaccount.dfs.core.windows.net/my_path

and the same config properties. I have got:

org.apache.hadoop.fs.azure.AzureException: No credentials found for account mycont@mybsaccount.dfs.core.windows.net in the configuration, and its container mycont is not accessible using anonymous credentials. Please check if the container exists first. If it is not publicly available, you have to provide account credentials.

I would like to know how to access Blob Storage using MSI authentication via Hadoop SDK.

Thanks, Sergiy

Sergey Shabalov
  • 23
  • 4
  • Please refer to https://hadoop.apache.org/docs/current/hadoop-azure/index.html. It seams that azure blob just support access key or sas token auth. – Jim Xu Nov 05 '20 at 01:06
  • 1
    Shure I have read this doc dozen times. But there is nothing about MSI. Seems like the WASB unit does support "Access Key" and SAS only :( – Sergey Shabalov Nov 05 '20 at 12:17
  • I think so. Do you have other concerns? if you have no, could I post it as an answer? – Jim Xu Nov 05 '20 at 12:54

0 Answers0