1

I was able to setup AutoScaling events as rules in EventBridge to trigger SSM Commands, but I've noticed that with my chosen Target Value the event is passed to all my active EC2 Instances. My Target key is a tag shared by those instances, so my mistake makes sense now.

I'm pretty new to EventBridge, so I was wondering if there's a way to actually target the instance that triggered the AutoScaling event (as in extracting the "InstanceId" that's present in the event data and use that as my new Target Value). I saw the Input Transformer, but I think that just transforms the event data to pass to the target.

Thanks!

EDIT - help with js code for Lambda + SSM RunCommand

I realize I can achieve this by setting EventBridge to invoke a Lambda function instead of the SSM RunCommand directly. Can anyone help with the javaScript code to call a shell command on the ec2 instance specified in the event data (event.detail.EC2InstanceId)? I can't seem to find a relevant and up-to-date base template online, and I'm not familiar enough with js or Lambda. Any help is greatly appreciated! Thanks

Sample of Event data, as per aws docs

{
  "version": "0",
  "id": "12345678-1234-1234-1234-123456789012",
  "detail-type": "EC2 Instance Launch Successful",
  "source": "aws.autoscaling",
  "account": "123456789012",
  "time": "yyyy-mm-ddThh:mm:ssZ",
  "region": "us-west-2",
  "resources": [
      "auto-scaling-group-arn",
      "instance-arn"
  ],
  "detail": {
      "StatusCode": "InProgress",
      "Description": "Launching a new EC2 instance: i-12345678",
      "AutoScalingGroupName": "my-auto-scaling-group",
      "ActivityId": "87654321-4321-4321-4321-210987654321",
      "Details": {
          "Availability Zone": "us-west-2b",
          "Subnet ID": "subnet-12345678"
      },
      "RequestId": "12345678-1234-1234-1234-123456789012",
      "StatusMessage": "",
      "EndTime": "yyyy-mm-ddThh:mm:ssZ",
      "EC2InstanceId": "i-1234567890abcdef0",
      "StartTime": "yyyy-mm-ddThh:mm:ssZ",
      "Cause": "description-text"
  }
}

Edit 2 - my Lambda code so far

'use strict'

const ssm = new (require('aws-sdk/clients/ssm'))()

exports.handler = async (event) => {
    const instanceId = event.detail.EC2InstanceId
    var params = {
        DocumentName: "AWS-RunShellScript",
        InstanceIds: [ instanceId ],
        TimeoutSeconds: 30,
        Parameters: {
          commands: ["/path/to/my/ec2/script.sh"],
          workingDirectory: [],
          executionTimeout: ["15"]
        }
    };

    const data = await ssm.sendCommand(params).promise()
    const response = {
        statusCode: 200,
        body: "Run Command success",
    };
    return response;
}
Merricat
  • 2,583
  • 1
  • 19
  • 27
  • you cannot really call shell command on the EC2 from the lambda – Sándor Bakos Aug 25 '21 at 20:46
  • what you can do is extract the data instanceId and trigger a run command with the instanceId as a parameter – Sándor Bakos Aug 25 '21 at 20:48
  • Yes, that's what I'm trying to do. I meant run a script that's already on the ec2 instance. – Merricat Aug 25 '21 at 21:09
  • well here is the API for it: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SSM.html#sendCommand-property – Sándor Bakos Aug 25 '21 at 21:13
  • I added the lambda code I've got so far – Merricat Aug 25 '21 at 21:18
  • Add the Parameters property inside the sendCommand option like: Parameters: { commands: [ 'echo something']} – Sándor Bakos Aug 25 '21 at 21:34
  • I've found a python example, but you got the idea, https://gist.github.com/lrakai/18303e1fc1fb1d8635cc20eee73a06a0 – Sándor Bakos Aug 25 '21 at 21:36
  • So I've updated the code I have worked with so far...I'm currently just using Lambda's test button, and the test is configured with the json sample event data posted above, changing the `EC2InstanceId` value with my own. The Execution results time out. I'm not too sure how `.promise()`, `await`, and `async` work, or if the syntax is correct. If I remove the `.promise()` I get success, but I don't think it's actually working. Wouldn't that trigger the script on my instance? Because it's not leaving a log as intended. – Merricat Aug 26 '21 at 03:28
  • Maybe I could have an issue with Roles and Permissions. I let lambda auto-generate and added an inline policy for SSM with Write:SendCommand. For its document and instance Resources, I ticked "Any in this account". Could the problem otherwise be with my VPC, subnets, or security group? any particular protocols/ports to keep open? – Merricat Aug 26 '21 at 03:32
  • what did the lambda log says?, the async await part looks fine, and try to log out the response from the ssm command, aka log the data variable that you have defined – Sándor Bakos Aug 26 '21 at 17:13
  • if i leave the `.promise()`, no logs, it just times out. If I remove it, and use `console.log(data)` right after the `sendCommand()`, I get a long thing that looks like the request data, like `useAccelerateEndpoint: false, clientSideMonitoring: false, endpointDiscoveryEnabled: undefined, endpointCacheSize: 1000, hostPrefixEnabled: true, stsRegionalEndpoints: 'legacy'`. I also see `operation: 'sendCommand', params: {` with all the params properly constructed – Merricat Aug 26 '21 at 17:55
  • what if you increase the lambda function timeout? for like 5 minutes? – Sándor Bakos Aug 26 '21 at 17:59
  • try to debug it and I would use the SSM Run Command from the console, and see if your script get executed – Sándor Bakos Aug 26 '21 at 18:01
  • I'm currently trying with 5 mins. would the Test trigger the script anyway? – Merricat Aug 26 '21 at 18:02
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/236454/discussion-between-merricat-and-sandor-bakos). – Merricat Aug 26 '21 at 18:10
  • Hey @SándorBakos so I've figured out that the main issue was that my javascript was still using a VPC, and the python one wasn't. As soon as I add the vpc to the lambda function, it starts doing the timeout again. Do you know why? I'm using the same vpc and subnets as the ec2 instances, and using the default security group (all ports). Is there a reason this lambda needs "internet access" to execute properly? I was wondering if technically the document "AWS-RunShellScript" is a resource outside my vpc – Merricat Aug 26 '21 at 23:26

2 Answers2

2

Yes, but through Lambda

EventBridge -> Lambda (using SSM api) -> EC2

Thank you @Sándor Bakos for helping me out!! My JavaScript ended up not working for some reason, so I ended up just using part of the python code linked in the comments.

1. add ssm:SendCommand permission:

After I let Lambda create a basic role during function creation, I added an inline policy to allow Systems Manager's SendCommand. This needs access to your documents/*, instances/* and managed-instances/*

2. code - python 3.9

import boto3
import botocore
import time

def lambda_handler(event=None, context=None):
    try:
        client = boto3.client('ssm')
    
        instance_id = event['detail']['EC2InstanceId']
        command = '/path/to/my/script.sh'
        
        client.send_command(
            InstanceIds = [ instance_id ],
            DocumentName = 'AWS-RunShellScript',
            Parameters = {
                'commands': [ command ],
                'executionTimeout': [ '60' ]
            }
        )
Merricat
  • 2,583
  • 1
  • 19
  • 27
1

You can do this without using lambda, as I just did, by using eventbridge's input transformers.

I specified a new automation document that called the document I was trying to use (AWS-ApplyAnsiblePlaybooks).

My document called out the InstanceId as a parameter and is passed this by the input transformer from EventBridge. I had to pass the event into lambda just to see how to parse the JSON event object to get the desired instance ID - this ended up being

$.detail.EC2InstanceID 

(it was coming from an autoscaling group).

I then passed it into a template that was used for the runbook

{"InstanceId":[<instance>]}

This template was read in my runbook as a parameter.

This was the SSM playbook inputs I used to run the AWS-ApplyAnsiblePlaybook Document, I just mapped each parameter to the specified parameters in the nested playbook:

 "inputs": {
      "InstanceIds": ["{{ InstanceId }}"],
      "DocumentName": "AWS-ApplyAnsiblePlaybooks",
      "Parameters": {
        "SourceType": "S3",
        "SourceInfo": {"path": "https://testansiblebucketab.s3.amazonaws.com/"},
        "InstallDependencies": "True",
        "PlaybookFile": "ansible-test.yml",
        "ExtraVariables": "SSM=True",
        "Check": "False",
        "Verbose": "-v",
        "TimeoutSeconds": "3600"
      }

See the document below for reference. They used a document that was already set up to receive the variable

https://docs.aws.amazon.com/systems-manager/latest/userguide/automation-tutorial-eventbridge-input-transformers.html

This is the full automation playbook I used, most of the parameters are defaults from the nested playbook:

 {
"description": "Runs Ansible Playbook on Launch Success Instances",
"schemaVersion": "0.3",
"assumeRole": "<Place your automation role ARN here>",
"parameters": {
  "InstanceId": {
    "type": "String",
    "description": "(Required) The ID of the Amazon EC2 instance."
  }
},
"mainSteps": [
  {
    "name": "RunAnsiblePlaybook",
    "action": "aws:runCommand",
    "inputs": {
      "InstanceIds": ["{{ InstanceId }}"],
      "DocumentName": "AWS-ApplyAnsiblePlaybooks",
      "Parameters": {
        "SourceType": "S3",
        "SourceInfo": {"path": "https://testansiblebucketab.s3.amazonaws.com/"},
        "InstallDependencies": "True",
        "PlaybookFile": "ansible-test.yml",
        "ExtraVariables": "SSM=True",
        "Check": "False",
        "Verbose": "-v",
        "TimeoutSeconds": "3600"
      }
    }
  }
]

}