6

I have a not completely orthodox CF->S3 setup. The relevant components here are:

  1. Cloudfront distribution with origin s3.ap-southeast-2.amazonaws.com

  2. Lambda@Edge function (Origin Request) that adds a S3 authorisation (version 2) query string (Signed using the S3 policy the function uses).

The request returned from Lambda is completely correct. If I log the uri, host and query string I get the file I am requesting. However, if I access it through the Cloudfront link directly, the request fails because it no longer uses the AWSAccessKeyID, instead it opts to use x-amz-cf-id (but uses the same Signature, Amz-Security-Token etc). CORRECTION: it may not replace, but be required in addition to.

I know this is the case because I have returned both the StringToSign and the SignatureProvided. These both match the Lambda response except for the AWSAccessKeyID which has been replaced with the x-amz-cf-id.

This is a very specific question obviously. I may have to look at remodelling this architecture but I would prefer not to. There are several requirements which has led me down this not completely regular setup.

Yves M.
  • 29,855
  • 23
  • 108
  • 144
danielgormly
  • 1,050
  • 1
  • 7
  • 16
  • 1
    `x-amz-cf-id` isn't an authorization header. It's a request identifier for internal troubleshooting. However, since it's an `x-amz-*` header, it may need to be included in the signing algorithm input, which is impossible since it isn't accessible to the Lambda function. Before I suggest a workaround, please clarify exactly how the request fails -- is it `SignatureDoesNotMatch`? Please show the error response XML including the string-to-sign, redacting only as necessary, if this is the error response. – Michael - sqlbot Mar 07 '18 at 05:45
  • Thanks Michael. Before I recreate the XML, I've tried again using AWS auth v4 as a query string (I've validated that the URL generated is fine) but this time I get back the more specific: `There were headers present in the request which were not signed x-amz-cf-id`. To answer your question though, yes it was `SignatureDoesNotMatch`. – danielgormly Mar 07 '18 at 06:03

4 Answers4

2

I believe the AWSAccessKeyID => x-amz-cf-id replacement is the result of two mechanisms:

First, you need to configure CloudFront to forward the query parameters to the origin. Without that, it will strip all parameters. If you use S3 signed URLs, make sure to also cache based on all parameters as otherwise you'll end up without any access control.

Second, CloudFront attaches the x-amz-cf-id to the requests that are not going to an S3 origin. You can double-check at the CloudFront console the origin type and you need to make sure it is reported as S3. I have a blog post describing it in detail.

But adding the S3 signature to all the requests with Lambda@Edge defeats the purpose. If you want to keep the bucket private and only allow CloudFront to access it then use the Origin Access Identity, that is precisely for the use-case.

Tamás Sallai
  • 3,195
  • 1
  • 14
  • 25
  • Your blog post helped a bunch for my attempt at proxying the S3 signed URLs through CloudFront. Thanks! My pain point was trying to get CDK to render a Cloudformation template that would show the origin as "S3" in the console. Here's my fix for that: `const s3Origin = new origins.S3Origin(pdfBucket, { originAccessIdentity: true as any});` `(s3Origin as any).origin.renderS3OriginConfig = () => ({});` – Charlie Laabs Nov 08 '21 at 16:47
1

I had alike task of returning S3 signed URL from a CloudFront origin request Lambda@Edge. Here is what I found:

If your S3 bucket does not have dots in the name you can use S3 origin in CloudFront, use domain name in the form of <bucket_name>.s3.<region>.amazonaws.com and generate signed URL e.g. via getSignedUrl from @aws-sdk/s3-request-presigner. CloudFront should be configured to pass URL query to the origin. Do not grant CloudFront access to S3 bucket in this case: presigned URL will grant access to the bucket.

However, when your bucket does have dots in the name, the signed URL produced by the function will have path-style URL and you will need to use CloudFront custom origin with s3.<region>.amazonaws.com domain. When using custom origin, CloudFront adds "x-amz-cf-id" header to the request to the origin. Quite inconveniently, value of the header should be signed. However, provided you do not change the origin domain in the Lambda@Edge return value, CloudFront seems to use the same value for "x-amz-cf-id" header as passed to the lambda event in event.Records[0].cf.config.requestId field. You can then generate S3 signed URL with the value of the header. With AWS JavaScript SDK v3 this can be done using S3Client.middlewareStack.add.

Here is an example of a JavaScript Lambda@Edge producing S3 signed URL with "x-amz-cf-id" header:

const {S3Client, GetObjectCommand} = require("@aws-sdk/client-s3");
const {getSignedUrl} = require("@aws-sdk/s3-request-presigner");

exports.handler = async function handler(event, context) {
    console.log('Request: ', JSON.stringify(event));

    let bucketName = 'XXX';
    let fileName = 'XXX';
    let bucketRegion = 'XXX';
    // Pre-requisite: this Lambda@Edge function has 's3:GetObject' permission for bucket ${bucketName}, otherwise you will get AccessDenied

    const command = new GetObjectCommand({
                                             Bucket: bucketName, Key: fileName,
                                         });

    const s3Client = new S3Client({region: bucketRegion});
    s3Client.middlewareStack.add((next, context) => async (args) => {
        args.request.headers["x-amz-cf-id"] = event.Records[0].cf.config.requestId;
        return await next(args);
    }, {
                                     step: "build", name: "addXAmzCfIdHeaderMiddleware",
                                 });

    let signedS3Url = await getSignedUrl(s3Client, command, {
        signableHeaders: new Set(["x-amz-cf-id"]), unhoistableHeaders: new Set(["x-amz-cf-id"])
    });


    let parsedUrl = new URL(signedS3Url);

    const request = event.Records[0].cf.request;
    if (!request.origin.custom || request.origin.custom.domainName != parsedUrl.hostname) {
        return {
            status: '500',
            body: `CloudFront should use custom origin configured to the matching domain '${parsedUrl.hostname}'.`,
            headers: {
                'content-type': [{key: 'Content-Type', value: 'text/plain; charset=UTF-8',}]
            }
        };
    }
    request.querystring = parsedUrl.search.substring(1);  //drop '?'
    request.uri = parsedUrl.pathname;

    console.log('Response: ', JSON.stringify(request));
    return request;
}
Yaegor
  • 1,771
  • 9
  • 12
0

So it seems like with Authentication V2 or V4, the x-amz-cf-id header that's appended to the origin request and inaccessible by the Lambda@Edge origin request function must be included in the authentication string. This is not possible.

The simple solution is to use the built-in S3 integration in Cloudflare, use a Lambda@Edge origin request function that switches the bucket if like me, that's your desired goal. For each bucket you want to use, add the following policy to allow your CF distribution to access the objects within the bucket.

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity <CloudfrontID>"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<bucket-name>/*"
        }
    ]
}

CloudfrontID refers to the ID under Origin Access Identity, not the Amazon S3 Canonical ID.

Yves M.
  • 29,855
  • 23
  • 108
  • 144
danielgormly
  • 1,050
  • 1
  • 7
  • 16
0

X-amz-cf-id is a reserved header of CF and it could be get by event as event['Records'][0]['cf']['config']['requestId']. You don't have to calculate Authentication V4 with X-amz-cf-id.