How Do I Enable Object-Level Logging for an S3 Bucket using boto3

Question

I'm trying to create an amazon cloudWatch rule which triggers whenever an object is uploaded into a bucket. I know that to do this I need to trigger on the PutObject Event, however best I can tell that requires enabling object level logging on the bucket. I will be using a multitude of buckets and want to be able to automate that process, and because of how most of the system is set up using boto3 seems to make the most sense. So how can I turn object-level logging on using boto3?

The only AWS official resource I've been able to find so far is: How Do I Enable Object-Level Logging for an S3 Bucket with AWS CloudTrail Data Events?

Which explains how to enable object level logging through the GUI. I've also looked through the boto3 library documentation

Both have ultimately not been helpful based on my understanding.

My chief goal is to enable object-level logging through boto3, if that's something that can be done.

Your chief goal is actually to fire the rule, isn't it? What will your rule be doing? I suspect you won't be able to trigger a CloudWatch rule on PutObject, since it doesn't go into the normal CloudTrail. Would it be acceptable to trigger an AWS Lambda function instead? — John Rotenstein, Jul 26 '19 at 15:32
@JohnRotenstein In the grand scheme of thing the team I'm on really wants to go the direction of a state machine as the ultimate executable, if you know of a way other than cloudwatch to trigger a state machine that'd be great, but to the best of my knowledge you can't just directly connect the s3 events and a state machine. — H. Pope, Jul 29 '19 at 12:21
What functionality are you seeking when you say you wish to "trigger a state machine"? Would an AWS Lambda function suffice? — John Rotenstein, Jul 29 '19 at 22:28
@JohnRotenstein the aim is to execute a state machine from AWS Step Functions, if that can be done through lambda then while it wouldn't be the ideal route it may have to do. — H. Pope, Jul 30 '19 at 12:30
Yes, you could use an AWS Lambda function to call Step Functions. Much easier and more immediate than using CloudTrail. — John Rotenstein, Jul 30 '19 at 23:37

score 0 · Accepted Answer · answered Jul 29 '19 at 22:27

0

You can configure an Amazon S3 Event so that, when a new object is created, it can:

Trigger an AWS Lambda function
Put a message in an Amazon SQS queue
Send a message to an Amazon SNS topic

See: Configuring Amazon S3 Event Notifications

answered Jul 29 '19 at 22:27

John Rotenstein

241,921
22
380
470

score 0 · Answer 2 · answered Jul 31 '20 at 22:57

You can use the put_event_selectors() function in CloudTrail service.

client = boto3.client('s3')
client.put_event_selectors(
TrailName='TrailName',
EventSelectors=[
    {
        'ReadWriteType': 'All',
        'IncludeManagementEvents': True,
        'DataResources': [
            {
                'Type': 'AWS::S3::Object',
                'Values': [
                    'arn:aws:s3:::your_bucket_name/',
                ]
            },
        ]            
    },
])

How Do I Enable Object-Level Logging for an S3 Bucket using boto3

2 Answers2