6

We can set up event rules to trigger an ECS task, but I don't see if the triggering event is passed to the runing ECS task and in the task how to fetch the content of this event. If a Lambda is triggered, we can get it from the event variable, for example, in Python:

def lambda_handler(event, context):
...

But in ECS I don't see how I can do things similar. Going to the cloudtrail log bucket doesn't sound to be a good way because it has around 5 mins delay for the new log/event to show up, which requires ECS to be waiting and additional logic to talk to S3 and find & read the log. And when the triggering events are frequent, this sounds hard to handle.

Z.Wei
  • 3,658
  • 2
  • 17
  • 28
  • related: https://stackoverflow.com/q/59206175/5599 – Anthony Sep 04 '20 at 16:20
  • 2
    Trigger an ECS job when an S3 upload completes With an example of sending the event content https://medium.com/@bowbaq/trigger-an-ecs-job-when-an-s3-upload-completes-3559c44c37d1 – Anthony Sep 07 '20 at 12:26

2 Answers2

6

One way to handle this is to set two targets In the Cloud watch rule.

  • One target will launch the ECS task
  • One target will push same event to SQS

So the SQS will contain info like

{
  "version": "0",
  "id": "89d1a02d-5ec7-412e-82f5-13505f849b41",
  "detail-type": "Scheduled Event",
  "source": "aws.events",
  "account": "123456789012",
  "time": "2016-12-30T18:44:49Z",
  "region": "us-east-1",
  "resources": [
    "arn:aws:events:us-east-1:123456789012:rule/SampleRule"
  ],
  "detail": {}
}

So when the ECS TASK up, it will be able to read event from the SQS. enter image description here

For example in Docker entrypoint

#!/bin/sh
echo "Starting container"
echo "Process SQS event"
node process_schdule_event.sj


#or if you need process at run time
schdule_event=$(aws sqs receive-message --queue-url https://sqs.us-west-2.amazonaws.com/123456789/demo --attribute-names All --message-attribute-names All --max-number-of-messages 1)
echo "Schdule Event: ${schdule_event}"


# one process done, start the main process of the container
exec "$@"
Adiii
  • 54,482
  • 7
  • 145
  • 148
4

After further investigation, I finally worked out another solution that is to use S3 to invoke Lambda and then in that Lambda I use ECS SDK (boto3, I use Python) to run my ECS task. By this way I can easily pass the event content to ECS and it is nearly real-time.

But I still give credit to @Adiii because his solution also works.

Z.Wei
  • 3,658
  • 2
  • 17
  • 28
  • Hi, can you elaborate on "easily pass the event content to ECS" - does this refer to the "Overrides" passed to "RunTask" to change the values of the Task. environment vars? – Anthony Sep 04 '20 at 15:19
  • 2
    Yes, in ecs.run_task(overrides={'containerOverrides': [ { 'environment': [ .... – Z.Wei Sep 09 '20 at 17:46