When I make API call for listing the paths in ADLS Gen2 using maxResults and Continuation as uri parameters.
Initially I'm getting the correct continuation token returned as response headers but when i try to make subsequent calls for listing the remaining files, sometimes ADLS returns with a continuation token that ends with "=="(lets call it type one) and sometimes if returns with the normal token(type two).Now,when I try to make API call using the first type, ADLS gives me an error and it works fine in the case of the other token.
I searched this before and what I found in one of the answers was that we have to ignore the response in which the token returned ends with "==" and we have to make calls again and again ignoring the continuation token(not using this token in the uri and using the same previous request uri) returned until we get the second(working) type of token.
This is a complex task when it comes to larger amount of files,is there a better solution for this? Is there a better solution for getting the right type of continuation token for making subsequent calls?
The Sample request for making the call is
GET https://storageAccountName.dfs.core.windows.net/sampleDirectory?recursive={recursive}&resource=filesystem&maxresults={maxresults}&continuation={continuation}
The two types of sample continuation tokens returned are
LCJhbGciOiJSUzI1NiIsIng1dCI6ImpTMVhvMU9XRGpfNTJ2Ynd==(Not working one)LCJhbGciOiJSUzI1NiIsIng1dCI6ImpTMVhvMU9XRGpfNTJ2Ynd(working one)
The Official Documentation for ADLS Gen2 for listing paths is https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/list
The link to the stackoverflow answer which I was referring to can't get ADLS Gen2 REST continuation token to work
