How to trigger a Lambda on S3 file upload

Question

I can't seem to get my lambda to trigger as I would expect. In AWS EventBridge, I created a rule with the following custom event pattern:

{
  "source": ["aws.s3"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": [
      "CopyObject",
      "CompleteMultipartUpload",
      "PutObject"
    ],
    "requestParameters": {
      "bucketName": ["my-bucket"],
      "key": [{"prefix": "folder1/folder2/"}]
    }
  }
}

but when I upload a file to the "directory" the rule does not trigger the lambda.

Can someone tell me where I've gone wrong?

you have enabled cloudtrail events for selected s3 bucket as documented [here](https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-cloudtrail-logging-for-s3.html#enable-cloudtrail-events)? — Balu Vyamajala, Mar 05 '21 at 04:57
Three things: 1. Why not use S3 triggers directly? 2. The `eventNames` might miss the `s3:` prefix (e.g. `s3:PutObject`) and 3. did you give EventBridge permission to invoke your Lamdba? — Jens, Mar 05 '21 at 07:13
@Jens - I think #3 might be the issue. Can you point me in the direction of docs that explain how to do that please? — maxmoore14, Mar 05 '21 at 21:01
I have tested your use case with cloud watch events, it works for me. Please find the below solution — Siva Sumanth, Mar 08 '21 at 05:05
1. s3 event notification is coupled with the resource, there are reasons to prefer eventbridge. My question: I can't get mine to work either, are regions relevent? I have us-east-1 bucket and us-west-2 target. Are event bus setting needed? I'm lost. — Merlin, Oct 08 '21 at 17:46

score 2 · Answer 1 · answered Mar 05 '21 at 10:21

2

You can use event notification in S3. It has direct integration with Lambda, SNS, SQS

Goto properties tab in S3
Navigate to Event notification. click on create event notification.
Then add the event name.
If you want to restrict the event to a specific folder or file type, you can fill in prefix or suffix fields or if you want it for entire bucket leave those blank.
Then select a list of events. like put, post.
Then choose the destination and click on save. You will be notified.

answered Mar 05 '21 at 10:21

Siva Sumanth

607
4
10

1

My requirement is to use EventBridge Rules. – maxmoore14 Mar 05 '21 at 21:02
Hi Maxmoore14 please find the below answer for the cloud watch event rule to put object – Siva Sumanth Mar 08 '21 at 04:05

score 2 · Answer 2 · answered Jan 24 '22 at 13:24

As of 2021-11-29, Amazon Event Bridge now supports Amazon S3 Event Notifications without having to enable CloudTrial.

In the properties of the S3 bucket you want to monitor, you have to enable Amazon EventBridge.

This will allow you to send messages to EventBridge using Rules.

In your eventbridge rule, you can configure lambda as a "Target". I made a step-by-step tutorial on how to configure eventbridge in AWS if you want to follow along: https://youtu.be/k-jEuNb_KBM

Siva Sumanth · Answer 3 · 2021-03-08T04:20:04.297

1

You can use both S3 events or cloud watch event rule to invoke the lambda function for any event on S3.

Below Image shows how to configure cloud watch event rule for s3 putObject operation.
Make sure you enabled cloud trail for data events in the respective region.
Make sure you create a rule for the specific bucket.
if you mention is for all the buckets. You will get unnecessary invocations, as cloud watch event rules works on cloud trail, where it will store all the logs to s3.

edited Mar 08 '21 at 04:20

answered Mar 08 '21 at 04:12

Siva Sumanth

607
4
10

Hope this is the answer you are expecting @maxmoore14 – Siva Sumanth Mar 08 '21 at 04:16
Seems like OP is trying to filter on a bucket prefix for events, is this also configurable in EventBridge? I couldn't find this. – Merlin Oct 08 '21 at 02:17

score 1 · Accepted Answer · answered Mar 10 '21 at 23:52

1

Thanks to all for the help and suggestions - all good resources should someone come across this post in the future. Turns out in my case @Balu Vyamajala was correct in the comments - I had incorrectly configured CloudTrail.

answered Mar 10 '21 at 23:52

maxmoore14

701
7
26

Please provide the solution, I'm still in the dark here. – Merlin Oct 08 '21 at 17:46

score 0 · Answer 5 · answered Mar 08 '21 at 09:42

As mentioned in my comment to your question before, you might not have the proper permissions to allow EventBridge to Invoke your Lambda.

You can add the following Resource-based policy to your Lambda:

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "AllowExecutionFromEventBridge",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "<lambda-arn>",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "<eventbridge-rule-arn>"
        }
      }
    }
  ]
}

You need to replace <lambda-arn> and <eventbridge-rule-arn> with the respective values.

You can read more about resource-based policy here: Using resource-based policies for AWS Lambda.

If you use tools like Terraform, you can use the following snippet:

resource "aws_lambda_permission" "example" {
  statement_id  = "AllowExecutionFromCloudWatch"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.example.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.example.arn
}

You need to replace function_name and source_arn here as well, with your specific references.

How to trigger a Lambda on S3 file upload

5 Answers5