8

When developing for Azure storage accounts, I can run the Microsoft Storage Emulator to locally keep Blobs, Queues, and Tables without having to connect to Azure online.

Is there something equivalent for the Azure Data Lake Store? It would be nice to develop locally for a while without having to connect to Azure online.

HaveSpacesuit
  • 3,572
  • 6
  • 40
  • 59

3 Answers3

1

Have you tried Visual Studio with the Azure Data Lake Tools plug-in?

David Paul Giroux
  • 657
  • 4
  • 10
  • Yes. In Visual Studio under Cloud Explorer I see a Data Lake Analytics node under (Local). Under actual subscriptions, I can see an actual Data Lake Store node. – HaveSpacesuit May 04 '17 at 19:06
1

As pointed out by David, you can develop Azure Data Lake Analytics (ADLA) projects locally without needing connectivity to Azure for the ADLA account or the associated Azure Data Lake Store (ADLS) account. Is there some other application you would like to use with ADLS?

Thanks, Sachin Sheth Azure Data Lake team

Sachin Sheth
  • 309
  • 1
  • 3
  • 1
    I don't _think_ I'm doing an analytics project, but this is a new area for me. For now, I'm trying to create directories and files in the Azure Data Lake Store. I've worked through the sample application [here](https://learn.microsoft.com/en-us/azure/data-lake-store/data-lake-store-get-started-net-sdk). I can see things I've uploaded to the data store by viewing the Data Explorer. What I'm trying to figure out is if it's possible to use emulated local storage instead of connecting to the actual online store. – HaveSpacesuit May 04 '17 at 19:48
0

Same problem here.

AFAIK the Storage Emulator is not yet able to really handle Data Lake (ADSL Gen2) Requests.
This Uri works (but looks for a file, not a dir):

http://127.0.0.1:10000/devstoreaccount1/packages-container/Dir/SubDir?sv=2020-04-08&se=2022-10-13T14%3A43%3A39Z&sr=b&sp=rcwl&sig=d2SxwYCkJGyx%2BHac9vntYQZOTt5QVs1bKgKb4%2FgcQ9k%3D

This one doesn't:
Error: Status: 403 (Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.) ErrorCode: AuthorizationFailure

http://127.0.0.1:10000/devstoreaccount1/packages-container/Dir/SubDir?sv=2020-04-08&se=2022-10-13T14%3A43%3A39Z&sr=d&sp=rcwl&sdd=2&sig=KU%2Fcu6W0Nsv8CucMgusubo8RbXWabFO8nDMkFxU1tTw%3D

The difference is that the second one uses the resource 'sr=d' (directory) while the first uses 'sr=b' (blob). Both items are working on real Azure Storage (with ADSL Gen2).

The request is already tracked here: https://github.com/Azure/Azurite/issues/553
Tested on VS 2022 17.3.6 using Server: Azurite-Blob/3.18.0

BerndK
  • 1,045
  • 10
  • 14
  • The OP is referring to Gen1 Data Lakes, rather than the Gen2 referenced in your answer here – iamdave May 03 '23 at 10:46
  • @iamdave : I can't see that explicitly Gen1 was requested, however he is asking for local emulated storage and is using folders (they are availabe Gen1 and 2 AFAIK). I was not able to use folders in emulated storage. – BerndK May 09 '23 at 20:42