Should I be getting a Cloudfront ‘MissingKey’ error when using an unsigned URL to access a public-read S3 object?

Question

Let's say I'm building a Dropbox clone: Filebox. I'm storing all my user's files with S3 and I'm using Cloudfront as my CDN.

So I've got a restricted S3 bucket (files.filebox.com) with a bucket policy that allows s3:GetObject to only the Origin Access Identity I created via Cloudfront. This forces all 'file' requests to go through Cloudfront and disallows anyone to access a file via the S3 URLs.

The Bucket Policy for files.filebox.com

{
    "Version": "2008-10-17",
    "Id": "AllowCloudfrontGet",
    "Statement": [
        {
            "Sid": "AllowCloudFrontGet",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity XXXXXXXXXXXX"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::files.filebox.com/*"
        }
    ]
}

The Unexpected Behaviour

Let's say, for simplicity sake, I'm storing all files with a canned ACL of public-read.

So this URL should work:
https://files.filebox.com/<userID>/some-file.txt

But this one should result in a 403: https://s3.amazonaws.com/files.filebox.com/<userID>/some-file.txt

But I'm seeing the opposite results. The S3 URL works fine, but the files.filebox.com URL, which is going through Cloudfront, is throwing a MissingKey error.

The files.filebox.com URL does work, but only if I sign the URL, even for objects that have an ACL of public-read.

Questions

Given that the bucket policy only allows s3:GetObject for the CF OAI, shouldn't the S3 URL fail with a 403, even if the object has a public-read ACL?
I can't find any information on this in the documentation other than vague language that seems to indicate that a restricted bucket should 403 for any requests that don't come through via Cloudfront.

When I set the ACL of an object to public-read does that not obviate the need to sign the Cloudfront URL, even on a restricted bucket?

Do I need to add another Statement to my Bucket Policy that allows unsigned URL access to public-read objects?
I attempted this, but it didn't work. I'm not sure how to write a Statement like this even ...

How do I truly deny any requests that come through via the S3 URLs?
I'd like to throw a 403 for any requests that come through on S3 URLs, even for objects with an ACL of public-read.

Answer 1

Given that the bucket policy only allows s3:GetObject for the CF OAI, shouldn't the S3 URL fail with a 403, even if the object has a public-read ACL?

No. public-read means public (unauthenticated) reads are allowed directly from S3. Because the bucket owner allowed the object to be uploaded with the public-read ACL, the object is public with the bucket owner's implicit consent, even with no bucket policy in place at all.

Object ACLs and bucket policy Allow options are additive. If you don't want the object to be accessible via the S3 endpoints, don't make it public.

When I set the ACL of an object to public-read does that not obviate the need to sign the Cloudfront URL, even on a restricted bucket?

No, it doesn't. CloudFront doesn't evaluate the object ACL. It decides based on the Cache Behavior settings whether to require authentication. If configured to require authentication for a given Path Pattern, it requires authentication, whether the entity behind CloudFront is S3 or something else. If CloudFront denies access, no request is even sent to S3.

Do I need to add another Statement to my Bucket Policy that allows unsigned URL access to public-read objects?

No. Nothing in the bucket policy modifies the behavior of CloudFront's front-end authentication.

The typical approach is to designate certain prefixes as public and others as not, and to configure CloudFront Cache Behaviors with appropriate, matching Path Patterns, so e.g. /public/* might be configured not require signed URLs, but /private/* might be require them.

This setting is called Restrict Viewer Access, and is set at the Cache Behavior level (thus at the Path Pattern level).

I'd like to throw a 403 for any requests that come through on S3 URLs, even for objects with an ACL of public-read.

Setting the object ACL to public-read means, by definiton, that the object is supposed to be accessible directly from the S3 bucket endpoint without credentials. Nothing more, nothing less.

In that light, what you are saying is analogous to saying "I'd like to prevent someone from stealing my car, even though I am planning on leaving the keys in the ignition." It can be done, but it's solving the wrong problem... and it gets complicated very quickly, because you have to resort to tactics like Deny with NotPrincipal conditions, to include exceptions not only for CloudFront, but also for yourself in the console as well as your applications, roles, etc., that access the bucket.

The point of the Origin Access Identity is so that CloudFront has a set of credentials that it can use when sending requests to the bucket, whether CloudFront requires the user to present authentication credentials, or not. With an OAI in place, there is no reason to use public-read on object ACLs.