2

I want to trigger a lambda whenever a new EC2 instance is registred in SSM's Fleet Manager (meaning the instance can be connected to using SSM), however I can't find what pattern to use in EventBridge.

Within EventBridge, I tried using the following pattern I found in the docs (so far its looks like the closest thing to my goal):

{
  "source": ["aws.ssm"],
  "detail-type": ["Inventory Resource State Change"]
}

However when I create a new EC2 and wait for its SSM agent to become active, it still doesn't trigger the above pattern.

Any idea how to catch this kind of event?

Mit94
  • 718
  • 8
  • 28
  • I couldn't find anything for that event in cloudtrail. I think you have to stream the SSM logs to Cloudwatch and create a notification system based on that – Paolo Nov 15 '22 at 18:33

1 Answers1

1

I think you have to go through CloudTrail API call. Please find below a CloudFormation template I used in the past that was working. Please note that it just provides the SSM resources. You need to add your own SQS queue as well (see SQS.ARN) and I've used the association with the tag registration set to enabled. So that if you have a lambda function connected, you can set it to false so if the instance connect again, it won't go to the same process again.

AWSTemplateFormatVersion: "2010-09-09"
Description: >
  SSM Registration event

# Description of the resources to be created.
Resources:
  RegistrationDocument:
    Type: AWS::SSM::Document
    Properties:
      DocumentType: Command
      Content:
        schemaVersion: "2.2"
        description: >
          An Automation Document ran by registered instances that gathers their software inventory
          and automatically updates their AWS SSM Agent to the latest version.
        mainSteps:
          - name: GatherSoftware
            action: aws:softwareInventory
          - name: Sleep
            action: aws:runShellScript
            inputs:
              runCommand:
                - sleep 20 || true
          - name: UpdateAgent
            action: aws:updateSsmAgent
            inputs:
              agentName: amazon-ssm-agent
              source: https://s3.{Region}.amazonaws.com/amazon-ssm-{Region}/ssm-agent-manifest.json
              allowDowngrade: "false"

  RegistrationDocumentAssociation:
    Type: AWS::SSM::Association
    Properties:
      AssociationName: !Sub registration-association-${AWS::StackName}
      Name: !Ref RegistrationDocument
      Targets:
        - Key: tag:registration
          Values:
            - enabled

  RegistrationEventRule:
    Type: AWS::Events::Rule
    Properties:
      Description: >
        Events Rule that monitors registration of AWS SSM instances
        and logs them to an SQS queue.
      EventPattern:
        source:
          - aws.ssm
        detail-type:
          - AWS API Call via CloudTrail
        detail:
          eventName:
            - UpdateInstanceAssociationStatus
          requestParameters:
            associationId:
              - !Ref RegistrationDocumentAssociation
            executionResult:
              status:
                - Success
      State: ENABLED
      Targets:
        - Arn: SQS.ARN
          Id: SqsRegistrationSubscription
          SqsParameters:
            MessageGroupId: registration.events
brushtakopo
  • 1,238
  • 1
  • 3
  • 16
  • Nope this didn't trigger my lambda. Actually I have seen all the events I had in Cloudtrail around the time when I am aunching a new EC2 (with an SSM Agent in it) and there isn't much about ssm (`ssm.amazonaws.com` as Event Source). Ocasionally there is an `UpdateInstanceInformation` as Event Name but that's all. – Mit94 Nov 15 '22 at 17:02
  • From what I recall, you need to create an SSM document (e.g updateSsmAgent). Then create a SSM Association with the previously created document. So when an new instance connects to SSM, the SSM document will be associated to the instance and execute. This is when you will see the event I mentioned in the response. I can try to give you a CloudFormation template tomorrow. – brushtakopo Nov 15 '22 at 21:18
  • I just did this. I associated the SSM document `AWS-UpdateSSMAgent` to all my instances. I can see the instances being listed in State Manager and that the associated SSM Document has been executed on it. However, I don't have any log on CloudTrail regarding this. The only events I have with `ssm.amazonaws.com` are `ListInstanceAssociations` and `UpdateInstanceInformation` and since they repeat at multiple times I can't use them (I need my lambda function triggered only once when the instance starts). I would gladly appreciate a CloudFormation template if you can provide one. – Mit94 Nov 16 '22 at 15:06
  • yes, I updated my response with the correct filter on CloudTrail API call. The filter is on the `UpdateInstanceAssociationStatus` with a `Success` result. – brushtakopo Nov 16 '22 at 15:16
  • I implemented it thanks to your template. However when I launch a instance there is a lot of these events happening, see : https://imgur.com/a/uWjRR92. Is this the expected behaviour? I thought I would only have 1 event each time the instance is being run. – Mit94 Nov 16 '22 at 16:43
  • 1
    there are because it has different status. You can expand each one of them. What you will get in your SQS queue is the event with status success. Please open a few of them and look at the content. – brushtakopo Nov 16 '22 at 16:53
  • Thanks for the clarification. However after multiple tries, I noticed that it was the "GatherSoftware" software step in the document that was sending the `UpdateInstanceAssociationStatus` events. If you leave only the "UpdateAgent" step you won't get those. This is also what I observed when I associated the SSM document `AWS-UpdateSSMAgent` with all my instances. – Mit94 Nov 17 '22 at 09:53
  • ok, but you do see only 1 success event right? This is the one that should be catched and push into SQS. – brushtakopo Nov 17 '22 at 10:00
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/249693/discussion-between-mit94-and-anthony-b). – Mit94 Nov 17 '22 at 10:04