AWS SAM YAML File not able to refer to an existing bucket for S3 event

Question

Use Case : Refer to an existing bucket while creating a S3 event for the Lambda function in SAM YAML file

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Sample SAM Template for sam-app

Globals:
    Function:
        Timeout: 900
        MemorySize: 2048
        Environment: 
          Variables:
            TABLE_NAME: "111"
            ENDPOINT_OVERRIDE: "222"

Resources:
  SomePull:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: target/demo-1.0.0.jar
      Handler: com.xxxx.run.LambdaFunctionHandler::handleRequest
      Runtime: java8
      Role: arn:aws:iam::aaaa:roaaaale/aaaa/lambdaExecution
      events:
          bucket: codedeploytestxxx
          event: s3:ObjectCreated:*
          rules:
            - prefix: uploads
            - suffix: .jpg
          existing: true

Reference : https://github.com/serverless/serverless/pull/6290

I tried couple of approaches , but still I am not able to refer to an existing bucket while creating an event , what configuration(s) I am missing from the above.

Error I get after executing the above script :

Invalid. property events not defined for resource of type AWS::Serverless::Function

Pat Myron · Answer 1 · 2020-02-05T04:51:01.300

That pull request is from a third party Serverless Framework while you're using the AWS Serverless Application Model (SAM)

Properties are case-sensitive:

AWS::Serverless::Function documentation

AWS::Serverless::Function.Events documentation

AWS::Serverless::Function S3 Event documentation

Converting that section to AWS Serverless Application Model (SAM) syntax leads to this error:

Resources:
  SomePull:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: target/demo-1.0.0.jar
      Handler: com.xxxx.run.LambdaFunctionHandler::handleRequest
      Runtime: java8
      Role: arn:aws:iam::aaaa:roaaaale/aaaa/lambdaExecution
      Events:
        Event1:
          Type: S3
          Properties:
            Bucket: codedeploytestxxx
            Events: s3:ObjectCreated:*
            Filter:
              S3Key:
                Rules:
                - Name: prefix
                  Value: uploads
                - Name: suffix
                  Value: .jpg

[cfn-lint] E0001: Error transforming template: Resource with id [SomePull] is invalid. Event with id [Event1] is invalid. S3 events must reference an S3 bucket in the same template.

AWS Serverless Application Model (SAM) has an issue about referencing an existing S3 bucket here

The information is good, but I found the answer hard to read at first because the middle section is extra detail that most people don't need at a high level. I think to make it clearer at the top you could mention that SAM hasn't caught up with Serverless Framework yet, so right now it's not possible to reference an exiting bucket directly. Still, this should be marked as the correct answer. Thanks — Miles, Mar 20 '20 at 07:33

score 1 · Answer 2 · answered Dec 22 '20 at 15:43

What worked for me was to remove the event from the lambda completely and then add a Notification Configuration in the S3 bucket, as well as a Lambda Permission.

So your lambda becomes:

SomePull:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: target/demo-1.0.0.jar
    Handler: com.xxxx.run.LambdaFunctionHandler::handleRequest
    Runtime: java8
    Role: arn:aws:iam::aaaa:roaaaale/aaaa/lambdaExecution

I imported the existing bucket into a separate stack and then updated that stack to contain the following:

  BucketTopicPolicy:
    Type: AWS::Lambda::Permission
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName: <my lambda function>
      Principal: 's3.amazonaws.com'
    DeletionPolicy: Delete

  Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: <my existing bucket>
      NotificationConfiguration:
         LambdaConfigurations:
            - Event: 's3:ObjectCreated:*'
              Function: <my lambda function>
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain

It might very well be that you can import the existing bucket first and then add both the lambda and the lambda permission to the stack and simply update it. (Note that you cannot import existing resources to a stack if the stack contains also new resources)

Thanks @kopelitsa With this suggestion I was able to achieve the desired deployment setup - dev team works on one side using AWS SAM and locally invoke the functions, and the AWS CDK on the other side to prepare/deploy infrastructure that gums up everything together. — dmytrops, Dec 14 '21 at 13:30

AWS SAM YAML File not able to refer to an existing bucket for S3 event

2 Answers2

Linked