How to set amazon S3 bucket policy to private to everyone except admin?

Question

I've got a bucket where i've accidently uploaded thousands of files with ACL to :public_read I would like all files to be unavailable except with a generated access URL.

I tried to create a bucket policy with deny all to everyone, and allow all to me.

It doesnt work and all files are forbidden even with a generated access URL :

http://s3.amazonaws.com/myBucket/myFile.pdf?AWSAccessKeyId=AKIAIZB2XTOJ6KYB5SCA&Expires=1331137308&Signature=zRfPOj4XFBrXhyqDZ5DpwJqsWs0%3D

{
    "Version": "2008-10-17",
    "Id": "Policy1331136935471",
    "Statement": [
        {
            "Sid": "Stmt1331136294179",
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::myBucket/*"
        },
        {
            "Sid": "Stmt1331136364169",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::6527...3775:root"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::myBucket/*"
        }
    ]
}

UPDATE :
i found reference to the default deny in the doc but the AWS Policy Generator has only 2 values "Allow" and "Deny" does anyone has the syntax for default deny ?

Thanks for your help

I thought you should be able to change your 'Deny' statement to use 'NotPrincipal' instead of 'Principal', e.g. `"NotPrincipal": { "AWS": "YOUR-ACCOUNT-NUMBER" }` but having tried the same thing myself it doesn't seem to work. — Rory, Sep 29 '14 at 18:04

score 13 · Answer 1 · answered Mar 07 '12 at 17:33

13

This is caused by the respective Evaluation Logic of the The Access Policy Language used with Bucket Policies:

The goal at evaluation time is to decide whether a given request should be allowed or denied. The evaluation logic follows several basic rules:

By default, all requests to use your resource coming from anyone but you are denied

An allow overrides any default denies

An explicit deny overrides any allows

The order in which the policies are evaluated is not important

[emphasis mine]

The page also provides an instructive flow chart and discussion [which] describe in more detail how the decision is made.

So your Deny "*" overrides your Allow "arn:aws:iam::6527...3775:root". As illustrated in the flow chart linked above, you can work around this by removing the explicit deny in favor of the default deny (please be aware of the potential subtleties when Using ACLs and Bucket Policies Together, which don't seem to apply to your use case though).

answered Mar 07 '12 at 17:33

Steffen Opel

63,899
11
192
211

Thanks for your answer, i found reference to the default deny in the doc but the AWS Policy Generator has only 2 values "Allow" and "Deny" do you know the syntax for default deny – vdaubry Mar 07 '12 at 18:02
1

@vdaubry: _Default Deny_ is an implicit default value automatically in place once you start using a bucket policy; therefore you do not need to do anything but removing the explicit deny currently in place and will gain the implicit default deny automatically. – Steffen Opel Mar 07 '12 at 23:35
2

Despite this having lots of upvotes it doesn't work because the individual objects have an object ACL of public read, so the Default Deny is overridden by the object ACL. It seems we need a way to explicitly deny access via bucket policy to anyone except the principal(s) that should have access. Having hit the same problem I haven't found a way to set the bucket policies to achieve this. – Rory Sep 29 '14 at 18:00

score 6 · Answer 2 · answered Mar 17 '15 at 16:38

The policy you were using does not work because the deny takes precedence over the allow, so all users are denied access. The correct way to do this is using the NotPrincipal policy element. It allows you to apply a policy to all principles except a specific list. Your policy should then be:

{
    "Version": "2008-10-17",
    "Id": "Policy1331136935471",
    "Statement": [
        {
            "Sid": "Stmt1331136294179",
            "Effect": "Deny",
            "NotPrincipal": {
                "AWS": "arn:aws:iam::6527...3775:root"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::myBucket/*"
        },
        {
            "Sid": "Stmt1331136364169",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::6527...3775:root"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::myBucket/*"
        }
    ]
}

Note that I don't think the allow is actually necessary because your account should have access to the files because it is the bucket/object owner who is granted access by default. Though that depends on the ACLs of your objects.

score 2 · Accepted Answer · edited Jan 03 '16 at 17:49

Ok, so as far as I understand if I only set allow to me then it should be deny to everyone else by default.

I tried to set that bucket policy but my files are still downloadable : just remove the access key id from the URL, like : http://s3.amazonaws.com/myBucket/myFile.pdf

My files ACL are still set as :public-Read so it seems that indeed I have a conflict between bucket policy and individual files ACL.

I gave up making all files private through bucket policy, if anybody ever needs to make a large number of files private inside a S3 bucket, here is the rake task I finally wrote :

  desc "Make all objects in S3 private"
  task :make_private  => :environment do
    require 'aws/s3'

    bucket_name = 'yourBucket'
    marker = ""

    AWS::S3::Base.establish_connection!(
      :access_key_id => "yourKey",
      :secret_access_key => "yourSecret"
    )

    #create the read-only by me policy
    owner_grant = ACL::Grant.new
    grantee = ACL::Grantee.new
    owner_grant.grantee = grantee
    owner_grant.permission = 'READ'
    grantee.type = "CanonicalUser"
    grantee.id = 'yourID'
    grantee.display_name = "yourName"


    # Iterate over all files inside bucket and apply the policy to each files
    loop do
      objects = Bucket.objects(bucket_name, :marker=>marker, :max_keys=>1000)

      marker = objects.last.key
      puts "new marker is \"#{marker}\""

      objects.each do |obj|
          policy = S3Object.acl(obj.key, bucket_name)
          policy.grants = [owner_grant]
          S3Object.acl(obj.key, bucket_name, policy)
      end
    end
  end

P.S: For info I have tried to change all files ACL using Firefox S3 Organizer or bucket explorer, none of them works if you have several hundreds of thousands of files, they just freeze.

ACLs are a legacy approach to access control with S3 and respectively limited, therefore it's recommended to drop ACL usage all together once you start using the advanced bucket and/or IAM policies for access control to avoid respectively confusing conflicts. Afterwards it should be possible to achieve any desired configuration in principle (though admittedly the details are sometimes not exactly easy to grasp). — Steffen Opel, Mar 08 '12 at 23:46
My need is pretty simple : all files inside my buckets must be private and accessible only through a generated access URL. I didn't manage to get this behaviour through policies, i did'nt find answer on SO or on amazon help forum, this is the only way i found to achieve this. I understand that this solution is not optimum but i really think there is a problem if something so simple is so complex to achieve.... — vdaubry, Mar 09 '12 at 11:31
Sad thing is that it has been two years and as far as I can tell the problem remains. I'm trying to make a bucket accessible only to my app (via access key) but I haven't found a way to do it. — Ege Ersoz, Jun 04 '14 at 03:32

score 1 · Answer 4 · answered Jan 10 '19 at 05:59

1

It's easier now.

Just set Remove public access granted through public ACLs to True

answered Jan 10 '19 at 05:59

Suraj Shrestha

728
11
19

How to set amazon S3 bucket policy to private to everyone except admin?

4 Answers4

Linked