I have CloudFront using an S3 origin. I'd like to be able to generate signed URLs for uploading values into S3, if possible through the CloudFront distribution.
I've been poring over the docs but I haven't found anything on this. I'd imagine that as long as the query parameters are passed along, but is this the case?
Is the reason is to provide low latency access to user uploading files to a central bucket, from all around the world ? In that case, I would recommend using S3 Transfer Acceleration instead.
You might want to use Transfer Acceleration on a bucket for various reasons, including the following:
You have customers that upload to a centralized bucket from all over the world.
You transfer gigabytes to terabytes of data on a regular basis across continents.
You are unable to utilize all of your available bandwidth over the Internet when uploading to Amazon S3.
https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html
Amazon S3 transfer acceleration is using Cloudfront under the scene.
Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.
Yes, you can use a CloudFront distribution to ingest S3 file uploads.
Depending on your setup, S3 Transfer Acceleration suggested in another answer may still be a better option, it's certainly better documented and has more implementation examples!
s3:GetObject access) as a starting point, we'll be adding/changing that to s3:PutObject anyway.
POST
XXXXX is the OAI ID and YOUR-TARGET-BUCKET is, well, your target bucket: "Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Sid": "1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity XXXXX"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::YOUR-TARGET-BUCKET/*"
}
]
}
s3:PutObject as an allowed action, but if you're setting up the distribution for content ingestion only (i.e. delivered through another distribution, or not publicly available at all) you could just replace the allowed action: "Action": "s3:PutObject",
Once the distribution is updated, you would be able to add files to your bucket by making a PUT request to https://your-distribution-url/desired-s3-key-name using Postman or something. Remember to use a signed url here if that's what you've configured.
Your client connections to the CF edge should be consistently faster, while anecdotally S3 acceleration speed comparison isn't always favorable.
If I understand AWS pricing correctly, you will be charged for "Regional Data Transfer Out to Origin" ($0.02-$0.16/GB depending on region) compared with S3 Transfer Acceleration charges ($0.04-$0.08/GB depending on region.)
I'd imagine that as long as the query parameters are passed along, but is this the case?
Not quite. CloudFront can be configured to pass through the query string parameters, but it still won't support this use case, because CloudFront injects an X-Amz-Cf-Id header into the request that it sends to S3. Because the header starts with X-Amz-*, S3 requires it to be signed... but this is impossible, since you have no way of knowing its value.
A CloudFront origin access identity can be given permission to write to a bucket, but this is also messy, because then any path CloudFront can read from can also be written to -- CloudFront signed URLs allow "access" -- whatever kind of access CloudFront itself has -- and can't be configured as read-only, if CloudFront, via the OAI, is allowed to write to the bucket.
Transfer Acceleration is a better solution since it uses the CloudFront network for transport, but understands native S3 signed URLs. Your argument related to wanting to use the domain name rather than the bucket name in the Lambda function signing the URLs breaks down... because you can't sign an S3 URL in any event, without actually knowing the bucket name.
