ADLS Gen2 REST API giving error with status code 400

Question

I was going through the list of REST API's for Azure Data Lake Storage Gen2 and facing problems when i call the api's from CLI/Postman.

i just started with simple get request path list api.

But response am getting as below

{"error":{"code":"InvalidHeaderValue","message":"The value for one of the HTTP headers is not in the correct format.\nRequestId:ecb96bb0-501f-0030-2578-65cf12000000\nTime:2020-07-29T07:20:35.6240747Z"}}

After searching found sample code to create directory/file and tried to create directory but got response like

Put Request

curl -i -X PUT -H "x-ms-version: 2020-07-29" -H "content-length: 0" -H "Authorization: Bearer $ACCESS_TOKEN" "https://$AccountName.dfs.core.windows.net/mydata?resource=filesystem"

Response for the above request

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
100   202  100   202    0     0    115      0  0:00:01  0:00:01 --:--:--   115HTTP/1.1 400 The value         
for one of the HTTP headers is not in the correct format.
Content-Length: 202
Content-Type: application/json;charset=utf-8
Server: Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 8d615c77-001f-0070-2b79-65c82a000000
Date: Wed, 29 Jul 2020 07:25:54 GMT

{"error":{"code":"InvalidHeaderValue","message":"The value for one of the HTTP headers is not in the correct format.\nRequestId:8d615c77-001f-0070-2b79-65c82a000000\nTime:2020-07-29T07:25:55.9048230Z"}}

For authentication i am using https://login.microsoftonline.com/{{tenantID}}/oauth2/token? and then capturing the bearer token in Test section using this below code.

pm.test(pm.info.requestName, () => {
    pm.response.to.not.be.error;
    pm.response.to.not.have.jsonBody('error');
});
pm.globals.set("bearerToken", pm.response.json().access_token);

pm.test(pm.info.requestname, ()=>{
    pm.response.to.not.be.error;
    pm.response.to.not.have.jsonBody('error');
})
pm.globals.set('BearerToken',("Bearer ").concat(pm.response.json().access_token));

can anyone tell me where and what is going wrong?

Updating question as requested with Screenshots.

After doing so many trials figured outfilesystem-list i am getting response but for path-list getting error saying AuthorizationPermissionMismatch with 403 status code.

FileSystem List:

Path List:

Headers for the failed one

What is the resource for your token? It should be `https://storage.azure.com` — Hong Ooi, Jul 29 '20 at 10:08
IN resource was using `https://management.azure.com/` and tried with your suggestion also but the response is same. — Prateek Naik, Jul 29 '20 at 10:23
Your resource should be `https://storage.azure.com`. Did I say this before? Why yes, yes I did — Hong Ooi, Aug 07 '20 at 12:33
@HongOoi tried with `https://storage.azure.com` still the same response. `{"error":{"code":"AuthorizationPermissionMismatch","message":"This request is not authorized to perform this operation using this permission.\nRequestId:8cb9734d-f01f-0090-19b8-6c4bb3000000\nTime:2020-08-07T12:44:57.8346082Z"}}` — Prateek Naik, Aug 07 '20 at 12:46
@HongOoi tell me how come it is working for `file system - list` API ? — Prateek Naik, Aug 07 '20 at 12:47

Ivan Glasenberg · Answer 1 · 2020-08-07T02:32:58.087

1

Update 0807:

Suppose you have already created a Service principal. Then follow the steps below:

Step 1: In azure portal, your ADLS Gen2 -> click on the "Access Control" -> click "Add" -> click "Add role assignment" -> in the "Add role assignment" popup, select "Contributor" for Role; then select your Service Principal name; at last, click Save button. The screenshot as below(Note, it may take a few minutes to take effect.):

Then add the "Storage Blob Data Reader" role:

Step 2: In postman, input this url for get request: https://login.microsoftonline.com/<tenant_id>/oauth2/token.

Then in Body -> select "form-data", and input the following keys and values:

grant_type:  client_credentials
client_id :  xxxxxxx
client_secret: xxxxxx
resource: https://storage.azure.com/

The screenshot as below:

At last, you can call the ADLS Gen2 rest api(note: the x-ms-date should be set to today). The screenshot as below:

Update 0730: How to use SAS token for authentication.

1.Nav to azure portal -> your "data lake storage gen2 account" -> then in the left pane, click on "Shared access signature" -> then select proper values -> at last, click the "Generate SAS and connection string" button. And please refer to the screenshot below:

2.Open postman, and there're something you should note:

1.for SAS token, please replace the first ? with &. Then place the SAS token at the end of the url. 2.if you're using SAS token, then you no need to use Authorization in the request header, please remove it.

Here is the screenshot:

Please let me know if you still have more issues.

Original answer:

There're several errors in your request.

Take Path - List for example, the x-ms-version should be 2018-11-09; and you should add the x-ms-date in the request header; and please remove the Content-Length in the request header.

I test it at my side, it works ok. Here is the test result:

Please let me know if you still have more issues.

edited Aug 07 '20 at 02:32

answered Jul 29 '20 at 08:50

Ivan Glasenberg

29,865
2
44
60

made changes as suggested by but still getting error `{ "error": { "code": "InvalidAuthenticationInfo", "message": "Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nRequestId:2ca97766-701f-0027-438d-656619000000\nTime:2020-07-29T09:49:04.0210844Z" } }` – Prateek Naik Jul 29 '20 at 09:50
@BenBean, can you please let me know how do you generate the authentication? it's the authentication issue. – Ivan Glasenberg Jul 29 '20 at 09:51
If you see the postman screenshot, in first file i am authenticating and saving the bearer token in variable. And passing the variable in the next request( 2nd file). If i hover on the variable {{bearerToken}} it is showing the value. – Prateek Naik Jul 29 '20 at 09:54
@BenBean, I mean how do you get the original bearer token? – Ivan Glasenberg Jul 29 '20 at 09:56
If am doing anything wrong in postman then why am i not getting any response here also `https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/list#code-try-0` – Prateek Naik Jul 29 '20 at 10:29
any help on this – Prateek Naik Jul 29 '20 at 16:56
@BenBean, do you mind using `sas token` for authentication? it's quite simple to be generated. – Ivan Glasenberg Jul 30 '20 at 01:26
and for the rest api "Try it" button, I did a test, no response. I guess there is something wrong with it. – Ivan Glasenberg Jul 30 '20 at 01:34
Can you tell me how to do? Instead of bearer token for just need to add SAS token in authentication right? – Prateek Naik Jul 30 '20 at 01:35
@BenBean, can you log into azure portal, and nav to your "data lake gen2" account? – Ivan Glasenberg Jul 30 '20 at 01:36
Yes i can access “data lake gen2“ – Prateek Naik Jul 30 '20 at 01:41
@BenBean, please see the updated section in the answer:). – Ivan Glasenberg Jul 30 '20 at 02:01
Using SAS token worked but why it is giving error with bearer token? Any idea – Prateek Naik Jul 30 '20 at 02:15
@BenBean, I'll give it a try, and update you later. – Ivan Glasenberg Jul 30 '20 at 02:17
@BenBean, do you know some programming language like c#? – Ivan Glasenberg Jul 30 '20 at 02:20
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/218860/discussion-between-ben-bean-and-ivan-yang). – Prateek Naik Jul 30 '20 at 02:22

ADLS Gen2 REST API giving error with status code 400

Updating question as requested with Screenshots.

FileSystem List:

Path List:

Headers for the failed one

1 Answers1

Linked