1

I am trying to use IBM Cloud Object Storage to store images uploaded to my site by users. I have this functionality working just fine.

However, based on the documentation here (link) it appears as though only one object can be downloaded from a bucket at a time.

Is there any way a list of objects could all be downloaded from the bucket? Is there a different approach to requesting multiple objects from a COS bucket?

Aabhas
  • 363
  • 1
  • 4
  • 9

3 Answers3

1

Via the REST API, no, you can only download a single object at a time. But most tools (like the AWS CLI, or Minio Client) allow downloading all objects that share a prefix (eg foo/bar and foo/bas). The IBM forks of the S3 libraries also are now integrated with Aspera, and can transfer large directories all at once. What are you trying to do?

Nick Lange
  • 857
  • 6
  • 8
  • I am querying a db to get a list of objects that the front end requires. I am then trying to download these objects from the COS. However, I don't want to make separate requests for each object. I really just need a way to download these multiple objects at one go (all in the same bucket) – Aabhas Sep 25 '18 at 22:03
0

According to S3 spec (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html), you can only download one object at a time.

r1j1m1n1
  • 345
  • 1
  • 4
0

There are various tools which may help to download multiple objects at a time from COS. I used AWS CLI tool to download and upload the objects from/to COS.
So install aws-cli tool and configure it by supplying access_key_id and secret_access_key here.

Recursively copying S3 objects to a local directory: When passed with the parameter --recursive, the following cp command recursively copies all objects under a specified prefix and bucket to a specified directory.

C:\Users\Shashank>aws s3 cp s3://yourBucketName . --recursive

for example: 

C:\Users\Shashank>aws --endpoint-url http://s3.us-east.cloud-object-storage.appdomain.cloud s3 cp s3://yourBucketName  D:\s3\  --recursive

In my case having endpoint based on us-east region and I am copying objects into D:\s3 directory.

Recursively copying local files to S3: When passed with the parameter --recursive, the following cp command recursively copies all files under a specified directory to a specified bucket. 

C:\Users\Shashank>aws s3 cp myDir s3://yourBucketName/ --recursive

for example: 

C:\Users\Shashank>aws --endpoint-url http://s3.us-east.cloud-object-storage.appdomain.cloud s3 cp D:\s3 s3://yourBucketName/ --recursive

I am copying objects from D:\s3 directory to COS.

For more reference, you can see the link here.

I hope it works for you.

Shashank
  • 709
  • 8
  • 16