AWS S3 get object using URL

Question

I have a collection of URLs that may or may not belong to a particular bucket. These are not public.

I'm using the nodejs aws-sdk to get them.

However, the getObject function needs params Bucket and Key separately, which are already in my URL.

Is there any way I can use the URL?

I tried extracting the key by splitting URL with / and getting bucket by splitting with .. But the problem is the bucket name can also have . and I'm not sure if key name can have / as well.

score 13 · Answer 1 · answered May 25 '17 at 00:27

The amazon-s3-uri library can parse out the Amazon S3 URI:

const AmazonS3URI = require('amazon-s3-uri')

try {
  const uri = 'https://bucket.s3-aws-region.amazonaws.com/key'
  const { region, bucket, key } = AmazonS3URI(uri)
} catch((err) => {
  console.warn(`${uri} is not a valid S3 uri`) // should not happen because `uri` is valid in that example 
})

score 1 · Answer 2 · answered Feb 02 '23 at 19:02

To avoid installing a package.

const objectUrl = 'https://s3.us-east-2.amazonaws.com/my-s3-bucket/some-prefix/file.json'

const { host, pathname } = new URL(objectUrl);

const [, region] = /s3.(.*).amazon/.exec(host)
const [, bucket, key] = pathname.split('/')

score 0 · Answer 3 · answered Oct 18 '21 at 12:10

use this module parse-s3-url to set the parameter for the getObject.

 bucket.getObject( parseS3Url('https://s3.amazonaws.com/mybucket/mykey'), (err:any, data:any) =>{
    if (err) {
      // alert("Failed to retrieve an object: " + error);
    } else {
      console.log("Loaded " + data.ContentLength + " bytes");
      // do something with data.Body
    }
  });

AWS S3 get object using URL

3 Answers3