0

I was taking a look at this documentation from Microsoft.

https://docs-microsoft-com.translate.goog/es-es/azure/storage/blobs/storage-quickstart-blobs-python?_x_tr_sl=es&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=nui,sc,elem

On which is said that to create a container, we should use

# Create a unique name for the container
container_name = str(uuid.uuid4())

# Create the container
container_client = blob_service_client.create_container(container_name)

I think its easy, so I try to use it in my code. as follow

client = BlobServiceClient.from_connection_string(connection_string)
all_containers = client.list_containers(include_metadata=True)
for container in all_containers:
    print(container['name'], container['metadata'])
    # print("==========================")
    container_client = client.get_container_client(container.name)
    # print(container_client)
    blobs_list = container_client.list_blobs()
    for blob in blobs_list:
        print(blob.name)
        print("==========================")
    # # ============================= TARGET =======================================

    # Target Client
    target_connection_string = ''
    target_account_key = ''
    source_container_name = source_container_name
    target_blob_name = blob.name
    target_destination_blob = str(container['name'])
    print(target_destination_blob)
    container_client = BlobServiceClient.create_container(target_destination_blob)

    # Create target client
    target_client = BlobServiceClient.from_connection_string(target_connection_string)
    container = ContainerClient.from_connection_string(target_connection_string, target_destination_blob)

    # Create new blob and start copy operation.
    # new_blob = client.get_blob_client(destination_container_name, blob_name)    
    new_blob = target_client.get_blob_client(target_destination_blob, target_blob_name)
    new_blob.start_copy_from_url(source_blob.url)
    print(source_blob.url)

The purpose is loop through all containers in storage A, and copy all containers and blobs and send them to storage B, recreating the same container name and blobs (basically a backup system)

At this point, as the containers does not exists in storage B, I need to make sure that I am checking if those storage does not exists, and if this is true to create them, otherwise it can process the copy. But for now I was just trying to create a container with the above code, but I get this error.

TypeError: create_container() missing 1 required positional argument: 'name'

As the name should be a string. I tried to cast to str but I get the same exact error.

According to the documentation, the parameter should be a string, but that does not seems to work.

UPDATE:

This is my updated version of the code:

from typing import Container
from azure.storage.blob import BlobClient, BlobServiceClient, ContainerClient
from azure.storage.blob import ResourceTypes, AccountSasPermissions
from azure.storage.blob import generate_account_sas    
from datetime import *




#================================ SOURCE ===============================
# Source Client
connection_string = '' # The connection string for the source container
account_key = '' # The account key for the source container
source_container_name = '' # Name of container which has blob to be copied
blob_name = '' # Name of the blob you want to copy
destination_container_name = '' # Name of container where blob will be copied



# Create client
client = BlobServiceClient.from_connection_string(connection_string) 



# Create sas token for blob
sas_token = generate_account_sas(
    account_name = client.account_name,
    account_key = account_key, 
    resource_types = ResourceTypes(object=True, container=True),
    permission= AccountSasPermissions(read=True,list=True),
    # start = datetime.now(),
    expiry = datetime.utcnow() + timedelta(hours=4) # Token valid for 4 hours
)

# Create blob client for source blob
source_blob = BlobClient(
    client.url,
    container_name = source_container_name, 
    blob_name = blob_name,
    credential = sas_token
)

client = BlobServiceClient.from_connection_string(connection_string)
all_containers = client.list_containers(include_metadata=True)
for container in all_containers:
    
    print(container['name'], container['metadata'])
    # print("==========================")
    container_client = client.get_container_client(container.name)
    # print(container_client)
    blobs_list = container_client.list_blobs()
    for blob in blobs_list:
        print(blob.name)
        print("==========================")
    # # ============================= TARGET =======================================

    # Target Client
    target_connection_string = ''
    target_account_key = ''
    source_container_name = source_container_name
    target_blob_name = blob.name
    target_destination_blob = str(container['name'])
    print(target_destination_blob)
    
    # Create target client
    target_client = BlobServiceClient.from_connection_string(target_connection_string)
    container = ContainerClient.from_connection_string(target_connection_string, target_destination_blob)
    container_client = target_client.create_container(target_destination_blob)
    # Create new blob and start copy operation.  
    new_blob = target_client.get_blob_client(target_destination_blob, target_blob_name)
    new_blob.start_copy_from_url(source_blob.url)

My expectation for this behaviour, is too loop over all containers and blobs in SOURCE and create a exact replica of containers and blobs in TARGET

Once of the issue, is if I run the code the first time, it create only the first storage, but no blob, if I run it the second time, I get the error

ErrorCode:ContainerAlreadyExists
CSharpRocks
  • 6,791
  • 1
  • 21
  • 27
Nayden Van
  • 1,133
  • 1
  • 23
  • 70

1 Answers1

0

You need to create target_client before you try to create the container and use this when you call create_container. Please try the code below (partial code):

# Target Client
target_connection_string = ''
target_account_key = ''
source_container_name = source_container_name
target_blob_name = blob.name
target_destination_blob = str(container['name'])
print(target_destination_blob)

# Create target client
target_client = BlobServiceClient.from_connection_string(target_connection_string)
container_client = target_client.create_container(target_destination_blob)
Gaurav Mantri
  • 128,066
  • 12
  • 206
  • 241
  • Yep that creates the container but only one, the first container. I have 2 containers, and I was pretty sure that this process will loop through all of the and create a new one for each container in the loop. mmmm – Nayden Van Sep 21 '21 at 14:23
  • What do you see for this: `print(container['name'], container['metadata'])`? Do you see both container names getting printed? – Gaurav Mantri Sep 21 '21 at 14:25
  • Yes I see both of the. Which I named `container1` and `container2` – Nayden Van Sep 21 '21 at 14:27
  • Please edit your question and include the latest code. I am sure its something very trivial. – Gaurav Mantri Sep 21 '21 at 14:28
  • I solved the issue with the loop. I am missing just the error that if the container already exists, it throws an error – Nayden Van Sep 21 '21 at 14:53
  • Yeah. You would need to put some kind of exception handling that would deal with this error. Surprisingly Azure Storage Python SDK does not include `create if not exists` kind of scenario which is supported in all other SDKs. – Gaurav Mantri Sep 21 '21 at 14:55
  • Yes I am trying to find out `create if not exists` – Nayden Van Sep 21 '21 at 15:25
  • Yep, I cannot find anything. Any advice on the best approach? I tried with `try` and `except`. This solves the error side, but on the other hand does not copy the files either – Nayden Van Sep 21 '21 at 15:38
  • do you mind if I talk to you in chat?please. – Nayden Van Sep 21 '21 at 15:49
  • Can we do it tomorrow please? It's pretty late for me here in India. Thanks. – Gaurav Mantri Sep 21 '21 at 17:03
  • Sure, thank you so so so much for your time and help – Nayden Van Sep 21 '21 at 17:06