3

I'm using the following Docker image for a MS Sql Server ...

version: "3.2"
services:

  sql-server-db:
    image: mcr.microsoft.com/mssql/server:latest
    ports:
      - 1433:1433
    env_file: ./tests/.my_test_env

How do I construct a Docker image to emulate the Azure Data lake (gen 2)? I interact with the data lake using a the Python 3 SDK, documented here -- https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-python .

Dave
  • 15,639
  • 133
  • 442
  • 830

1 Answers1

1

I do not think this it is possible to create a Docker image which emulates ADLS Gen 2. ADLS Gen 2 is purely on Azure and there is no local emulator. Gen 1 did offer a local emulator, accessible via Visual Studio which was handy for ADLA projects. I think the article you mention is describing "installing libraries to interact with ADLS Gen" 2 as opposed to "installing libraries to act as ADLS Gen 2 emulator" I believe.

This article describes the Azure Storage Emulator and mentions Blob, Queue and Table services. ADLS Gen 2 is not mentioned, even though it is in a way a Blob account with the hierarchical namespace option enabled.

Do you have to use Gen 2, eg you could use emulated blob store for small volumes as per the article below or why not just use a real data lake account?

https://learn.microsoft.com/en-us/azure/storage/common/storage-use-emulator

wBob
  • 13,710
  • 3
  • 20
  • 37