Upload zarr folder to Azure blob container

Question

I tried to upload a zarr file (folder-like) to an Azure container using Python but it does not work properly, as it only uploaded the innermost files and deleted everything else in the container. This is my code:

def upload_zarr(file_path):
    credential = env_vars['STORAGE_ACCOUNT_KEY']
    container_client = ContainerClient(account_url='https://<account-name>.blob.core.windows.net',
                                      container_name='data',
                                      credential=credential)
    store = zarr.ABSStore(client=container_client, prefix='/')
    ds = zarr.open(file_path)
    zarr.save(store, ds)
    print("finished writing to Azure")

I don't think recursively upload the whole file as a folder recursively works since it erases the .zarr properties.

score 0 · Accepted Answer · answered Aug 28 '23 at 06:24

Upload zarr folder to Azure blob container

You can use the zarr.convenience.copy() method to Upload the zarr file to the Azure blob storage.

Code:

import zarr
from azure.storage.blob import ContainerClient 

def upload_zarr(file_path):
    credential = env_vars['STORAGE_ACCOUNT_KEY']
    container_client = ContainerClient(account_url='https://<account-name>.blob.core.windows.net',
                                      container_name='test',
                                      credential=credential)
    store = zarr.ABSStore(client=container_client, prefix='/')
    ds = zarr.open(file_path)
    group = zarr.group(store=store)
    zarr.convenience.copy(ds, group, 'sample.zarr')
print("finished writing to Azure")

Output: