9

I need to move vhds from one subscription to other. I would like to know which one is better option for the same: Start-AzureStorageBlobCopy or AzCopy?

Which one takes lesser time ?

Aatif Akhter
  • 2,126
  • 1
  • 25
  • 46

5 Answers5

12

Both of them would take the same time as all they do is initiate Async Server-Side Blob Copy. They just tell the service to start copying blob from source to destination. The actual copy operation is performed by Azure Blob Storage Service. The time it would take to copy the blob would depend on a number of factors including but not limited to:

  • Source & destination location.
  • Size of the source blob.
  • Load on storage service.
Gaurav Mantri
  • 128,066
  • 12
  • 206
  • 241
  • Gaurav can u plz share any link in support of your answer. – Aatif Akhter Apr 27 '15 at 06:28
  • Async Copy Blob: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-asynchronous-cross-account-copy-blob.aspx. AzCopy: http://blogs.msdn.com/b/windowsazurestorage/archive/2013/04/01/azcopy-using-cross-account-copy-blob.aspx. HTH. – Gaurav Mantri Apr 27 '15 at 06:35
2

Running AzCopy without specifying the option /SyncCopy and running PowerShell command Start-AzureStorageBlobCopy should take the same duration, because they both use server side asynchronous copy.

If you'd like to copy blobs across regions, you'd better consider specifying the option /SyncCopy while executing AzCopy in order to achieve a consistent speed because the asynchronous copying of data will run in the background of servers that being said you might see inconsistent copying speed among your “copying” operations.

If /SyncCopy option is specified, AzCopy will download the content to memory first, and then upload content back to Azure Storage. In order to achieve better performance of /SyncCopy, you are supposed to run AzCopy in the VM whose region is the same as source storage account. Besides that, the VM size (which decides bandwidth and CPU core number) will probably impact the copying performance as well.

For further information, please refer to Getting Started with the AzCopy Command-Line Utility

Zhaoxing Lu
  • 6,319
  • 18
  • 41
2

They don't take the same time.

I've tried to copy from one account to another and have a huge difference.

Start-AzureStorageBlobCopy -SrcBlob $_.Name -SrcContainer $Container -Context $ContextSrc -DestContainer $Container -DestBlob $_.Name -DestContext $ContextDst --Verbose

This takes about 2.5 hours.

& .\AzCopy.exe /Source:https://$StorageAccountNameSrc.blob.core.windows.net/$Container /Dest:https://$StorageAccountNameDst.blob.core.windows.net/$Container /SourceKey:$StorageAccountKeySrc /DestKey:$StorageAccountKeyDst /S

This takes several minutes.

I have about 600 Mb and about 7000 files here.

Elapsed time:            00.00:03:41
Finished 44 of total 44 file(s).
[2017/06/22 17:05:35] Transfer summary:
-----------------
Total files transferred: 44
Transfer successfully:   44
Transfer skipped:        0
Transfer failed:         0
Elapsed time:            00.00:00:08
Finished 345 of total 345 file(s).
[2017/06/22 17:06:07] Transfer summary:
-----------------
Total files transferred: 345
Transfer successfully:   345
Transfer skipped:        0
Transfer failed:         0
Elapsed time:            00.00:00:31

Do anyone know why it's so different?

Evgeniy
  • 564
  • 5
  • 14
  • 1
    That's a meaningful comparison. Appreciate your effort. I am also eager to know the reason. – Aatif Akhter Jun 22 '17 at 14:29
  • Maybe someone from MSFT can shed light on it? – Evgeniy Jun 22 '17 at 15:12
  • 1
    The difference between the two is likely `AzCopy` is operating a bulk mode and `Start-AzureStorageBlobCopy` is operating serially due to piping the blobs in one by one, sending request to initiate the transfer. i.e. `Get-AzureStorageBlob | Start-AzureStorageBlobCopy`. This would mean many individual network requests, triggering the copies. It is unclear if `Start-AzureStorageBlobCopy` can accept bulk array transfer like `AzCopy`. – PotatoFarmer Jun 01 '18 at 21:58
1

In most scenarios, AzCopy is likely to be quicker than Start-AzureStorageBlobCopy due to way you would initiate the copy resulting in fewer calls to Azure API:

  • [AzCopy]1 call for whole container (regardless of blob count)

vs

  • [Start-AzureStorageBlobCopy] N number of calls due to number of blobs in container.

Initially I thought it would be same as both appear to trigger same asynchronous copies on Azure side, however on client side this would be directly visible as @Evgeniy has found in his answer.

In 1 blob in container scenario, theoretically both commands would complete at same time.

*EDIT (possible workaround): I was able to decrease my time tremendously by:

  1. Removing console output AND
  2. Using the -ConcurrentTaskCount switch, set to 100 in my case. Cut it down to under 5 minutes now.
PotatoFarmer
  • 2,755
  • 2
  • 16
  • 26
0

AzCopy offers an SLA which the Async copy services lacks. AzCopy is designed for optimal performance. Use the/SyncCopy parameter to get a consistent copy speed.

user9276374
  • 1