How to retrieve multiple image from Amazon S3 using imgURL at once?

Question

I want to retrieve list of images in one go from Amazon S3 based on image URL.

Currently I am able to fetch single image using the following code:-

     AWS.config.update({
                    accessKeyId: accessKeyId,
                    secretAccessKey: secretAccessKey
                });

                AWS.config.region = region;

                var bucketInstance = new AWS.S3();
                var params = {
                    Bucket: bucketName,
                    Key: awsImgUrl
                }
                bucketInstance.getObject(params, function (err, file) {
                    if (file) {
                        var dataSrc = "data:" + file.ContentType + ";base64," + EncodeData(file.Body);
                        callbackSuccess(dataSrc);
                    } else {
                        callbackSuccess("Error");
                    }
                });

EncodeData = function (data) {
        var str = data.reduce(function (a, b) { return a + String.fromCharCode(b) }, '');
        return btoa(str).replace(/.{76}(?=.)/g, '$&\n');
    }

In my scenario I have multiple S3 image url like awsImgUrl1, awsImgUrl2..awsImgUrln.

How to fetch it in one go instead of one by one?

score 2 · Answer 1 · answered Oct 13 '18 at 19:44

You cannot get more than one image per api call with S3. You can however make multiple calls in parallel. Using promises this is straightforward.

var bucketInstance = new AWS.S3();
var imageKeys = [ awsImgUrl1, awsImgUrl2, awsImgUrl3];

var promisesOfS3Objects = imageKeys.map(function(key) {
  return bucketInstance.getObject({
    Bucket: bucketName,
    Key: key
  }).promise()
    .then(function (file) {
      return "data:" + file.ContentType + ";base64," + EncodeData(file.Body);
    })
  })
Promise.all(promisesOfS3Objects)
.then(callbackSuccess) // callbackSuccess is called with an array of string
.catch(function() { callbackSuccess("Error") })

score 1 · Answer 2 · answered Oct 10 '18 at 15:28

AWS SDK does not have any method to read multiple files as once and same with console, you can not download multiple files at once.

they have only GetObject method do read a object in bucket by key only.

so in your case you have to read one by one with their key name only if you already have key names as list..

you can get summary of objects in bucket if you would like to get list of objects then put a loop to download all files.

score 1 · Accepted Answer · edited Oct 17 '18 at 14:52

You can change the way you upload the image data. Instead of uploading a single image, upload one document containing multiple image datas.

const addImageBlock = () => {
  var photoBlock = [
    {
      imageId: 'id',
      type: 'png',
      body: 'data:image/png;base64,iVBORw0K...'
    },
    {
      imageId: 'id2',
      type: 'png',
      body: 'data:image/png;base64,iVBORw0K...'
    },
    {
      imageId: 'id3',
      type: 'png',
      body: 'data:image/png;base64,iVBORw0K...'
    },
    {
      imageId: 'id4',
      type: 'png',
      body: 'data:image/png;base64,iVBORw0K...'
    }
    //...ect
    ];
  s3.upload({
    Key: photoBlockId + '.json',
    Body: photoBlock,
    ACL: 'public-read'
  }, function(err, data) {
    if (err) {
      return alert('There was an error', err.message);
    }
  });
}

Then when you receive this data with one s3 call, you can loop through and render the images on the frontend,

getObject(params, function (err, file) {
   imageArr = [];
    if (file) {
       JSON.parse(file.toString()).map((image) => {
      var image = new Image();
      image.src = image.body;
      imageArr.push(image)
    })
       callbackSuccess(imageArr);
                    } 
    else {
      callbackSuccess("Error");
         }
});

How to retrieve multiple image from Amazon S3 using imgURL at once?

3 Answers3