0

What is the best way to copy blobs of one container to another container using java api? I know that I can list container and then perform copy for every blob which is listed, but when I'd like to copy entire container from ARCHIVE tier to HOT tier this could take up to 15 hours. So what would be the best options to copy containers between tiers (archive -> hot|cold) while also having in mind the reliability and fault tolerance?

Note: I don't want to rehydrate the container, I just want to leave it where it is and only create copy of it for processing. I've found azure-storage-blob-batch where it's possible to change the tier of all blobs, but there's no copy.

I'm missing some concept of copy container to another container in azure java SDK with posibilities of progress monitoring.

Thanks

bilak
  • 4,526
  • 3
  • 35
  • 75
  • The basic method is to use the rest api to send the request to the server. Or you can find the java method based on rest api from [this link](https://azuresdkdocs.blob.core.windows.net/$web/java/azure-storage-blob/12.0.0/index.html). – Cindy Pau Apr 27 '20 at 08:17

1 Answers1

1

I think the way is to use azure rest api. you can have a look of this doc:

https://learn.microsoft.com/en-us/rest/api/storageservices/copy-blob-from-url

All operations performed on Azure are essentially request sent by the client and processed by the server.

You can write put requests using java according to the above document, which can achieve your idea.

Of course, operations like copyblob should be able to find the API from below link, you can look at the following documentation: (Find the ones related to blobclient)

https://azuresdkdocs.blob.core.windows.net/$web/java/azure-storage-blob/12.0.0/index.html
Cindy Pau
  • 13,085
  • 1
  • 15
  • 27
  • do we have an sdk library for this or do I need to create som client by myself? Currently I' m using `azure-storage` with version `8.6.3` – bilak Apr 27 '20 at 10:04
  • @bilak Yes, we have a sdk in java. If you use java API, you need to download `com.azure.storage.blob`. If you use rest api, just add `x-ms-copy-source`, `x-ms-access-tier` as header and add SAS token to the url is enough. Please notice that when we copy the ARCHIVE tier blob, the target blob must also be in the same storage account. Take a look at this: https://learn.microsoft.com/en-us/rest/api/storageservices/copy-blob#remarks – Cindy Pau Apr 28 '20 at 01:27
  • Yes I figured this out by reading through docs. When you copy blob from archive to hot it remains in archive tier for some time (while rehydrating - docs says it could take 15 hours). Are you somehow monitoring the rehydrate process or do you know how to be notified when blob is rehydrated? – bilak Apr 28 '20 at 06:14
  • @bilak There is no specific method for the feature you mentioned, but we can use the characteristics of archive blob. When the blob is in the archive state, it does not accept access and modification operations, so you can write code to send a request to access the blob at intervals, and you can notify yourself when the access request is successful. :) – Cindy Pau Apr 28 '20 at 06:57
  • yes I was thinking about azure service bus utilization. So when blob is copied fire event (defferable) and check periodically whether rehydration is done, and if it's done fire another event if needed. – bilak Apr 28 '20 at 09:36
  • @bilak Hi, bilak. If my answer helps you find the solution, can you mark it as the answer to end this question? Thanks.:) – Cindy Pau Apr 29 '20 at 01:24
  • @BowmanZhu What about copying to another container? I want to copy from archive to cold in another container. – Gina Marano Nov 16 '20 at 07:25