Python 3 Boto 3, AWS S3: Get object URL

Question

I need to retrieve an public object URL directly after uploading a file, this to be able to store it in a database. This is my upload code:

   s3 = boto3.resource('s3')
   s3bucket.upload_file(filepath, objectname, ExtraArgs={'StorageClass': 'STANDARD_IA'})

I am not looking for a presigned URL, just the URL that always will be publicly accessable over https.

Any help appreciated.

score 32 · Accepted Answer · answered Feb 07 '18 at 22:03

32

There's no simple way but you can construct the URL from the region where the bucket is located (get_bucket_location), the bucket name and the storage key:

bucket_name = "my-aws-bucket"
key = "upload-file"

s3 = boto3.resource('s3')
bucket = s3.Bucket(bucket_name)
bucket.upload_file("upload.txt", key)
location = boto3.client('s3').get_bucket_location(Bucket=bucket_name)['LocationConstraint']
url = "https://s3-%s.amazonaws.com/%s/%s" % (location, bucket_name, key)

answered Feb 07 '18 at 22:03

ofrommel

2,129
14
19

4

This answer is wrong, you're not quoting the key properly. Also fails for new buckets that must be addressed using a domain name. – Antti Haapala -- Слава Україні Feb 07 '21 at 17:51
3

Now URL is changed to f"{bucket_name}.s3.{location}.amazonaws.com/{key}" – dolgom Aug 25 '22 at 12:34

lionels · Answer 2 · 2021-06-03T14:07:34.603

32

Since 2010 you can use a virtual-hosted style S3 url, i.e. no need to mess with region specific urls:

url = f'https://{bucket}.s3.amazonaws.com/{key}'

With quoted key :

url = f'''https://{bucket}.s3.amazonaws.com/{urllib.parse.quote(key, safe="~()*!.'")}'''

Moreover, support for the path-style model (region specific urls) continues for buckets created on or before September 30, 2020. Buckets created after that date must be referenced using the virtual-hosted model.

See also this blog post.

edited Jun 03 '21 at 14:07

answered May 11 '19 at 12:53

lionels

782
7
12

This does not work for GovCloud (and I assume for China) – raylu Nov 12 '19 at 03:48
Can you please explain a bit more ? (For china it's amazonaws.com.cn I guess) – lionels Nov 14 '19 at 15:36
This relies on the s3.amazonaws.com endpoint figuring out which region your bucket is in. That works for the aws partition, but it doesn't know anything about the GovCloud and China partitions. For GovCloud, it looks like s3-us-gov-west-1.amazonaws.com. – raylu Nov 15 '19 at 17:57
They should have this implemented in the api. It makes no sense to expect people to dig into details of which server at which time expects which format – MuhsinFatih Jun 27 '20 at 12:48
Doesn't quote key properly. – Antti Haapala -- Слава Україні Feb 07 '21 at 17:52
If your key contains spaces, S3 replaces those with '+'. A better approximation might be to replace `quote` with `quote_plus` in the above. – Peter Harrison Jul 15 '22 at 07:09

score 6 · Answer 3 · answered Oct 01 '20 at 13:58

6

Concatenating the the raw key will fail for some special characters in the key(ex: '+'), you have to quote them:

url = "https://s3-%s.amazonaws.com/%s/%s" % (
    location,
    bucket_name,
    urllib.parse.quote(key, safe="~()*!.'"),
)

Or you can call:

my_config = Config(signature_version = botocore.UNSIGNED)
url = boto3.client("s3", config=my_config).generate_presigned_url(
    "get_object", ExpiresIn=0, Params={"Bucket": bucket_name, "Key": key}
)

...as described here.

answered Oct 01 '20 at 13:58

Julian Go

4,442
3
23
28

1

Please note that `https://s3-us-east-1...` does not work, other regions do. To have the same behaviour you should use `https://s3.us-east-1...` (note the . instead of - ) See https://docs.aws.amazon.com/general/latest/gr/s3.html – Martin Roy Aug 24 '21 at 19:19

score 3 · Answer 4 · answered Jun 04 '21 at 11:14

You can generate a presigned URL and then trim its query parameters. This requires the "s3:PutObject" permission for the relevant bucket.

url = s3client.generate_presigned_url(ClientMethod = 'put_object',
                                      Params = { 'Bucket': bucket_name, 'Key': key })

# trim query params
url = url[0 : url.index('?')]

score -2 · Answer 5 · answered Apr 11 '19 at 18:33

Just a small note. The function call

location = 
    boto3.client('s3').get_bucket_location(Bucket=bucket_name['LocationConstraint']

may return location = None if the bucket is in the region 'us-east-1'. Therefore, I'd amend the above answer and add a line below that line:

if location == None: location = 'us-east-1'

Python 3 Boto 3, AWS S3: Get object URL

5 Answers5

Linked