AWS Lambda : Cross account Policy for Lambda function S3 to S3 copy

Question

we are trying to implement the lambda function which will copy the object from one S3 to another S3 bucket in cross account based on the source S3 bucket events. Currently we are able to copy the file between source and target within same SAG . But when we tried to implement the same logic with cross account , getting the CopyObject operation: Access Denied issue . I have given following bucket policy. Can you please help me to get the correct IAM and bucket policy to resolve this issue .

{
    "Version": "2012-10-17",
    "Id": "Policy1603404813917",
    "Statement": [
        {
            "Sid": "Stmt1603404812651",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::6888889898:role/Staff"
            },
            "Action": [
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::source-bucktet-testing-lambda/*",
                "arn:aws:s3:::source-bucktet-testing-lambda"
            ]
        }
    ]
}

based on the https://www.lixu.ca/2016/09/aws-lambda-and-s3-how-to-do-cross_83.html link , Yes, we can implement the same logic with help of access ID and access secret keys for source and dest. But am trying to implement same logic instead of access ID and access secret keys for source and dest, granting access for both source and target buckets with appropriate policy and make it work as like same account .

Is this bucket policy being applied to the Source bucket, or the Destination bucket? Is the `Staff` role assigned to the Lambda function? Please edit your question to include the permissions assigned to the role. — John Rotenstein, Oct 23 '20 at 03:02
SAG - Service account Group. Each team will have individual SAG account which has own privileges. — Karthikeyan Rasipalay Durairaj, Oct 23 '20 at 16:22
Is this bucket policy being applied to the Source bucket, or the Destination bucket? Is the Staff role assigned to the Lambda function? - Yes , This is policy being applied to the Source bucket. — Karthikeyan Rasipalay Durairaj, Oct 23 '20 at 16:23

John Rotenstein · Answer 1 · 2020-10-24T00:25:21.423

To reproduce your situation, I did the following:

In Account-A:
- Created an Amazon S3 bucket (Bucket-A)
- Created an IAM Role (Role-A)
- Created an AWS Lambda function (Lambda-A) and assigned Role-A to the function
- Configured an Amazon S3 Event on Bucket-A to trigger Lambda-A for "All object create events"
In Account-B:
- Created an Amazon S3 bucket (Bucket-B) with a bucket policy (see below)

IAM Role

Role-A has the AWSLambdaBasicExecutionRole managed policy, and also this Inline Policy that assigns the Lambda function permission to read from Bucket-A and write to Bucket-B:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucket-a/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::bucket-b/*"
        }
    ]
}

Bucket Policy on destination bucket

The Bucket Policy on Bucket-B permits access from the Role-A IAM Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT-A:role/role-a"
            },
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::bucket-b/*"
        }
    ]
}

Lambda Function

Lambda-A is triggered when an object is created in Bucket-A, and copies it to Bucket-B:

import boto3
import urllib

TARGET_BUCKET = 'bucket-b'

def lambda_handler(event, context):
    
    # Get incoming bucket and key
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    source_key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'])

    # Copy object to different bucket
    s3_resource = boto3.resource('s3')
    copy_source = {
        'Bucket': source_bucket,
        'Key': source_key
    }
    target_key = source_key # Change if desired

    s3_resource.Bucket(TARGET_BUCKET).Object(target_key).copy(copy_source, ExtraArgs={'ACL': 'bucket-owner-full-control'})

I grant ACL=bucket-owner-full-control because copying objects to buckets owned by different accounts can sometimes cause the objects to still be 'owned' by the original account. Using this ACL grants ownership to the account that owns the destination bucket.

Testing

I uploaded a file to Bucket-A in Account-A.

The file was correctly copied to Bucket-B in Account-B.

Comments

The solution does NOT require:

A bucket policy on Bucket-A, since Role-A grants the necessary permissions
Turning off S3 Block Public Access, since the permissions assigned do not grant 'public' access

Thanks for your response John . Can you please policy statement for IAM role and bucket policy — Karthikeyan Rasipalay Durairaj, Oct 23 '20 at 06:49
@JohnRotenstein this is a great explanation, tested and working like a charm. One thing that bothers me: what if you upload multiple files? I know that invocation is async. Another question is would the 3 second timeout be enough when files are bigger than certain size? Because of the async invocation my gut feeling is it does not matter, but what about a file e.g. >100MB? — Marko E, Dec 30 '21 at 09:59
@JohnRotenstein can you please share why the bucket policy is required for the dest bucket since the same has already been defined in the IAM role's policy? — Vinay, Jul 21 '22 at 18:58
@Vinay Let's say that you have an S3 bucket (`Bucket-B`) in your own AWS Account. Do you want people in other AWS Accounts to be able to write to it, with you incurring storage costs and potentially having your own data overwritten? Buckets are private by default and the Bucket Policy above says which other AWS Accounts or users can `PutObject` to _your_ bucket. Similarly, it stops me writing a policy in my Account to read from _your_ bucket. — John Rotenstein, Jul 21 '22 at 21:56
@JohnRotenstein That is a great point, thank you for the explanation. On the other hand, why is it necessary to have `PutObject` in the IAM role's policy if it is defined in the destination bucket's policy? I have a similar use case and necessary bucket policies in both source and destination buckets. Still, it didn't work until I attached a similar policy to lambda's IAM role. — Vinay, Jul 21 '22 at 22:34
@Vinay An IAM Role has no permission by default. If the Lambda function wants to call an API, it must be given permission to do so. The Bucket Policy is additionally required because the owner of the bucket is permitting access by a Role/User from a _different account_. Feel free to create a new Question if you want a deeper explanation. — John Rotenstein, Jul 21 '22 at 22:43

score 0 · Answer 2 · answered Oct 23 '20 at 04:19

Assuming the following

Above mentioned policy is for the source bucket
6888889898 is the Destination AWS account
Lambda for copying the file is located in the destination AWS account and has Staff role attached to it.

Even after setting all these correctly, the copy operation may fail. This is because the Policy allows you to get/put s3 objects, but not the tags associated with those s3 objects.

You will need to ALLOW the following actions as well "s3:GetObjectTagging" and "s3:PutObjectTagging"

AWS Lambda : Cross account Policy for Lambda function S3 to S3 copy

2 Answers2

Linked