-3

I am trying to send a curl request to an S3 bucket from my EC2 to retrieve a specific object within the bucket:

I want to create a transparent proxy with caching implemented by nginx so the aws cli wont work for this.

The EC2 instance (Linux machine) works as a proxy server with NGINX to send HTTP requests to the bucket for caching purposes, I do not have an SSL cert on this instance.

The bucket only contains images.

The curl request looks like this:

curl my-bucket.s3.eu-west-1.amazonaws.com/1450/1349/5467_1012.jpg

But I get an Access Denied error

I have attached a full read access policy to my EC2 instance role.

Here is my bucket policy:

{
    "Version": "2012-10-17",
    "Id": "MediaStorageBucketPolicy",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::${current_account}:root"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "${media_storage_bucket_arn}",
                "${media_storage_bucket_arn}/*"
            ]
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::${current_account}:root"
            },
            "Action": [
                "s3:PutObject"
            ],
            "Resource": [
                "${media_storage_bucket_arn}",
                "${media_storage_bucket_arn}/*"
            ]
        },

    {
        "Effect": "Allow",
        "Principal": {
            "AWS": [
                "arn:aws:I am::<account number>:role/ssm-ec2-service-role"
            ]
        },
        "Action": [
            "s3:GetObject",
            "s3:ListBucket"
        ],
        
        "Resource": [
            "${media_storage_bucket_arn}",
            "${media_storage_bucket_arn}/*"
        ]
    }


    ]
}

Can this be achieved without making the bucket public?

Tegue Morrison
  • 21
  • 4
  • 1
    If you wish to keep the objects in the Amazon S3 bucket 'private', then your request for the object will need to include authentication information. This could either be done by generating an authorization signature as part of the request, or by generating an [Amazon S3 pre-signed URL](https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html), which provides time-limited access to a private object. How do you want to prove to S3 that you are entitled to access the object? – John Rotenstein Sep 23 '22 at 09:05
  • @JohnRotenstein, ok great so: Without using SSL cert or being able to use the aws cli the only way for the instance to be allowed to read from the bucket would be: 1. Updating the bucket policy to allow anyone, even if they’re not signed in to AWS, to access your resource (this is of course, is not recommended). 2. generating an authorization signature as part of the request. 3. Use an Amazon S3 pre-signed URL, which provides time-limited access to a private object. Correct? – Tegue Morrison Sep 23 '22 at 10:31
  • Please provide enough code so others can better understand or reproduce the problem. – Community Sep 23 '22 at 11:05
  • 1
    Yes, that is correct. The first option makes it public, the other options provide proof to Amazon S3 that you are authorized to access the object. There are also other methods you could use (eg the bucket policy can limit to a specific IP address or limit to a specific referer) but they are less secure. – John Rotenstein Sep 23 '22 at 12:03

1 Answers1

0

The solution I found was creating an IAM user with S3 read access that can then be used within the EC2 (credentials of the user) to pull data from the bucket, see the following doc:

http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html

Tegue Morrison
  • 21
  • 4