Moving Azure blob storage files from one folder to another

Question

I'm trying to copy files from one folder to another in the same blob container (IE: everything in container/processing to container/archived). The following code creates the target folder and files but it's always 0 bytes. What is wrong with this code? I've tried using just a single stream, same problem. _blobServiceClient is the type BlobServiceClient. If I download and serialize the object to a C# class and then re-upload it, it works. But I'd prefer to make this as efficient as possible since I'm just moving things around in BlobStorage and don't actually need to use the contents in my code.

var containerClient = _blobServiceClient.GetBlobContainerClient(container);
var blobHierarchyItems = containerClient.GetBlobsByHierarchy(BlobTraits.None, BlobStates.None, null, sourceFolder);

foreach (var bhi in blobHierarchyItems)
{
    using (var downStream = new MemoryStream())
    using (var upStream = new MemoryStream())
    {
        var blobClient = containerClient.GetBlobClient(bhi.Blob.Name);
        blobClient.DownloadTo(downStream);
        containerClient.UploadBlob($"{targetFolder}/{bhi.Blob.Name}", upStream);
        downStream.Position = 0;
        await downStream.CopyToAsync(upStream);
    }
}

I've also tried using a BlockBlobClient.StartCopyFromUriAsync function. The results are the same, it creates the target folder and blob but no data. Here's that code:

var containerClient = _blobServiceClient.GetBlobContainerClient(container);
var blobHierarchyItems = containerClient.GetBlobs(BlobTraits.None, BlobStates.None, sourceFolder);
var targetBlockClient = containerClient.GetBlockBlobClient(targetFolder);

foreach (var bhi in blobHierarchyItems)
{
    var blobUri = new Uri($"{containerClient.Uri}/{bhi.Name}");
    var copyOp = await targetBlockClient.StartCopyFromUriAsync(blobUri);
    await copyOp.WaitForCompletionAsync();
}

Source folder

Target folder

Why are you downloading and uploading? Can’t you simply copy the blob from source to destination? — Gaurav Mantri, Jun 05 '20 at 16:17
It looks like you can use [`StartCopyFromUriAsync`](https://learn.microsoft.com/en-us/dotnet/api/azure.storage.blobs.specialized.blobbaseclient.startcopyfromuriasync?view=azure-dotnet) and [`WaitForCompletionAsync`](https://learn.microsoft.com/en-us/dotnet/api/azure.storage.blobs.models.copyfromurioperation.waitforcompletionasync?view=azure-dotnet) to perform a copy that doesn't go through your machine. — Kirk Larkin, Jun 05 '20 at 16:33
The reason for the problem you describe is that you upload `upStream` _before_ you copy `downStream` into it. — Kirk Larkin, Jun 05 '20 at 16:35
I don't want to download and upload, just copy would be ideal.. just haven't found anything that works yet.. unfortunately the api's have changed quite drastically from asp.net core 2->3 so having trouble finding the right way to do it — Ryan Langton, Jun 05 '20 at 16:43
@KirkLarkin I tried StartCopyFromUriAsync originally and believe I got an error.. will try again and post results — Ryan Langton, Jun 05 '20 at 16:44
Just one thing to keep in mind: No such thing as "folders" in Azure Blob Storage (unless you enable hierarchical storage). The folder name is just part of the blob's name. — David Makogon, Jun 05 '20 at 17:45
@KirkLarkin I updated the question after trying StartCopyFromUriAsync — Ryan Langton, Jun 05 '20 at 18:31
@KirkLarkin actually StartCopyFromUriAsync is working, it's just not keeping the full path for the source and copying that path to the target.. I'm looking into fixing that — Ryan Langton, Jun 05 '20 at 18:35
@KirkLarkin this works, thanks var targetBlockClient = containerClient.GetBlockBlobClient($"{targetFolder}/{bhi.Name}"); var blobUri = new Uri($"{containerClient.Uri}/{bhi.Name}"); — Ryan Langton, Jun 05 '20 at 18:51

Moving Azure blob storage files from one folder to another

0 Answers0