17

I have an S3 bucket with more than 100 million objects in it an each object has a unique key as usual. I was wondering if there is a way to assign another key to some of these objects. Something like this:

Key1 ---> Object1

Key2 ---> Object2

Key3 ---> Object2 (I'd like to add this)

I looked this up in AWS documentation but couldn't find anything relevant.

Another approach could be to create a dummy object with Key3 that redirects to actual object. Would that be easier?

Seckin Tozlu
  • 547
  • 2
  • 5
  • 16
  • why doing this? could you add a database, e.g dynamodb, to map one or several keys to an object in S3? What you try to achieve is not possible as far as I know – Tom Jan 27 '16 at 16:35
  • I could but we're trying to avoid having to lookup a database to determine mappings. If I could generate additional keys to the same file in advance I won't have to maintain anything. – Seckin Tozlu Jan 27 '16 at 16:58
  • Is your content publicly-accessible? If not, are you using pre-signed URLs or a different mechanism? Are you using http or https? As you can see from the comments, there **is** a workaround with empty dummy objects and a `x-amz-website-redirect-location` header that does what's intended... however, whether you can use this natively with no outboard componentry depends on how you're using the bucket, and there are some workarounds if you are not, that I'll need to work out details for in order to craft a good answer. – Michael - sqlbot Jan 28 '16 at 19:20
  • The content is publicly accessible via https. However my bucket is not enabled for web site hosting since we're not hosting web pages. I'm not sure if enabling web site hosting (if possible) for this purpose would be a good idea. – Seckin Tozlu Jan 28 '16 at 20:28
  • @SeckinTozlu - if is already available via HTTPS insn't this already a static website? Can you provide more info on your use case perhaps? – Rodrigo Murillo Jan 28 '16 at 22:25
  • @RodrigoM Not really, my bucket contains static images and documents that are publicly available to anyone but no html. My access pattern is "https://.s3.amazonaws.com/" but this doesn't look like the website endpoints that are documented in the aws developer guide. So I believe that I'm not using web site endpoints. – Seckin Tozlu Jan 28 '16 at 23:35
  • Got it. And how do you/users actually consume/see those images? Via a standard browser or app? – Rodrigo Murillo Jan 28 '16 at 23:44
  • Via a standard browser. – Seckin Tozlu Jan 29 '16 at 16:17

1 Answers1

17

A key is a unique identifier for an object:

From Object Key and Metadata

Each Amazon S3 object has data, a key, and metadata. Object key (or key name) uniquely identifies the object in a bucket.

Another approach could be to create a dummy object with Key3 that redirects to actual object. Would that be easier?

Yes you could do this: you could use User Defined Object Metadata to create Key3, and in the Metadata define a key/value pair like this:

x-amz-meta-KeyAlias=Key2

That Metadata expresses that this key is an alias for another key.

Your app could then read that metadata, and then redirect the request from Key3 to Key2.

If the bucket is exposed as a public website, you could also implement some form of redirection. UPDATE: This works for any object type/extension, when using the Website Endpoint

Rodrigo Murillo
  • 13,080
  • 2
  • 29
  • 50
  • 1
    According to this document: http://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-redirect.html you can have amazon redirect one page to another if you have a website endpoint by setting a special header. This functionality doesn't exist for rest endpoints? – Seckin Tozlu Jan 27 '16 at 17:14
  • 3
    As far as I know, that is only for S3 buckets that are websites, and only for .html pages. – Rodrigo Murillo Jan 27 '16 at 17:18
  • Are your objects mostly webpages? – Rodrigo Murillo Jan 27 '16 at 17:26
  • No, unfortunately they are .jpg, .png, .gif images and .pdf, ms office documents. – Seckin Tozlu Jan 27 '16 at 17:27
  • It does not appear that Website Redirect Location metadata can be used to redirect image assets – Rodrigo Murillo Jan 27 '16 at 17:42
  • Yes, it can. Requests have to be sent to the web site endpoint, and the static web site hosting feature has to be enabled, and the objects have to be publicly accessible, but for the case where all of these are met, it will redirect any request to another url inside or outside the bucket. – Michael - sqlbot Jan 27 '16 at 23:15
  • @Michael-sqlbot - can you confirm please? I am testing image file redirection, where https://bucket/test1.jpg has Website Redirect Location metadata /test2.jpg. When I open the original URL via curl or browser; I get a 200 response, not a 301. It is not redirecting to /test2.jpg for me. I have an open support ticket with AWS on this. Can you confirm if in your understanding, this type redirection setup/object should work? Thank you. – Rodrigo Murillo Jan 27 '16 at 23:50
  • 1
    Tested and verified, this works exactly as expected. `HTTP/1.1 301 Moved Permanently` `Location: /target-object-path-shown-here`. Remember that web site hosting has to be enabled and the request has to be sent to the `bucket.s3-website-[region].amazonaws.com` endpoint. The REST endpoint ignores the `x-amz-website-redirect-location` header. – Michael - sqlbot Jan 28 '16 at 19:16
  • Thanks @Michael-sqlbot that is correct. I was using the S3 object link endpoint, and not the API/s3-website endpoint. If you wish to answer with this approach. I will delete my answer. Can you explain why you call the S3 object link the REST endpoint? I understood that the 'bucket.s3-website-[region].amazonaws.com' was/is the API/REST endpoint, and that is why only this endpoint handles the redirection correctly. – Rodrigo Murillo Jan 28 '16 at 22:29
  • 2
    Your terminology is a little bit off... the REST/API endpoint is `bucket.s3[-region].amazonaws.com`, and the web site endpoint is `bucket.s3-website-[region].amazonaws.com` (for buckets with the website hosting feature enabled). See http://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteEndpoints.html#WebsiteRestEndpointDiff for confirmation of the terminology and the differences between the feature-sets of the two endpoints. – Michael - sqlbot Jan 28 '16 at 22:55
  • @Michael-sqlbot : I want to add multiple metadata in CreatePresignedPost , but not able to do . can you please suggest Conditions: [ ['content-length-range', 0, process.env.UPLOAD_MAX_FILE_SIZE], ['starts-with', '$Content-Type', contentType], { "x-amz-meta-uploaded-by": userId }, { "x-amz-meta-filename": Filenma } ] }; here I am passing two key pair values in metadata but it gives me error . – Akshay Champavat Apr 11 '21 at 08:04