Get Amazon S3 to redirect to my endpoint on missing object requests?

Question

We use S3 for online storage of files. To reduce costs, many of those files can be generated on demand with a path known ahead of time.

Is it possible to get S3 bucket to redirect missing object requests to my pre-configured endpoint (which can then generate and serve the file on demand)?

For example, a request to http://bucket1.s3.amazonaws.com/path2/file3.jpg would temporarily redirect (307) to http://mydomain.com/missing_s3_obj/bucket1/path2/file3.jpg.

You can always have your primary URL be your endpoint, which can verify whether the files exist or not in S3 before properly redirecting your client to S3 or generating the file on demand. Does it make sense? — Viccari, Jun 05 '12 at 01:47

score 6 · Accepted Answer · edited Oct 05 '16 at 10:07

You can use RoutingRules to have S3 redirect to your domain as described in the Example 3 of Configure a Bucket for Website Hosting from the S3 documentation. I believe it describes your situation accurately.

<RoutingRules>
  <RoutingRule>
  <Condition>
    <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
  </Condition>
  <Redirect>
    <HostName>mydomain.com</HostName>
    <ReplaceKeyPrefixWith>missing_s3_obj/bucket1/</ReplaceKeyPrefixWith>
    <HttpRedirectCode>307</HttpRedirectCode>
  </Redirect>
  </RoutingRule>
</RoutingRules>

score 1 · Answer 2 · answered Oct 22 '12 at 11:05

At the moment this isn't possible, although it is a very good idea. (Perhaps you should list it as a feature request on the S3 forum, where feature requests are taken actually very seriously).

What you want can be acheived using a different archiectecture set up though.

For example, rather than using S3, you would use Cloudfront instead. Your edge origin would be your own application serving up requests. You would have to store a hash of all available files on S3, but if the file is available on S3 you would proxy to S3. If it is not, then you would generate the file dynamically and serve that to Cloudfront.

You would then get all of the benefits of cachine for subsequent requests.

Get Amazon S3 to redirect to my endpoint on missing object requests?

2 Answers2

Linked