21

We want to create ScheduledTasks in AWS ECS via CloudFormation. Is there a programmatic way to create via boto or cloudformation?

siliconsenthil
  • 1,380
  • 1
  • 14
  • 25
  • We use the aws web console to create Scheduled Tasks. I could not find anything for Scheduled Task in troposphere (https://github.com/cloudtools/troposphere/blob/master/troposphere/ecs.py) – siliconsenthil Jun 13 '17 at 17:18
  • 1
    @siliconsenthil I usually takes couple of weeks for AWS to release the CLI as well as Cloudformation. They release it on console first. – AnkitG Jun 14 '17 at 17:47
  • @siliconsenthil Did you ever figure how to accomplish this with CF? I'm running into the same issue. – jckdnk111 Sep 19 '17 at 13:40
  • We did what is given as the accepted answer by Mark – siliconsenthil Sep 27 '17 at 13:22

3 Answers3

30

In order to define a scheduled ECS task in CloudFormation, you need to define a "AWS::Events::Rule" resource with an ECS task as a target.

"TaskSchedule": {
  "Type": "AWS::Events::Rule",
  "Properties": {
    "Description": "dump data every workday at 10",
    "Name": "dump-data",
    "ScheduleExpression": "cron(0 10 ? * MON-FRI *)",
    "State": "ENABLED",
    "Targets": [
      {
        "Id": "dump-data-ecs-task",
        "RoleArn": {
          "Fn::GetAtt": ["TaskSchedulerRole", "Arn"]
        },
        "EcsParameters": {
          "TaskDefinitionArn": {
            "Ref": "TaskDefinition"
          },
          "TaskCount": 1
        },
        "Arn": {
          "Fn::GetAtt": ["EcsCluster", "Arn"]
        }
      }
    ]
  }
}
Mark
  • 361
  • 3
  • 5
  • 3
    This, though you also need to specify an `Id` for your `Target` element. – Fiona Hopkins Oct 12 '17 at 13:50
  • 1
    And "Fn::GetAttr" should be "Fn::GetAtt" as well, but otherwise this is a great answer. – Nick Palmer Dec 27 '17 at 23:19
  • 1
    where is the actual cron task defined. I mean where is the function that is dumping the data defined? – awm Mar 05 '19 at 14:28
  • @awm Little late to reply here, but the targets themselves are responsible for the execution logic. For example, if you're using an ECS task as the target, the event will spin up a Task from its TaskDefinition in a given Cluster (which will then do whatever it was designed to do). – SamuelMS Feb 17 '21 at 02:59
6

Troposphere now allows to define ECS scheduled task, you need three resources for that,

  1. the task definition:

    from troposphere.ecs import (
    ContainerDefinition,
    TaskDefinition,
)

scheduled_worker_task_definition = TaskDefinition(
    "ScheduledWorkerTask",
    template=template,
    ContainerDefinitions=[
        ContainerDefinition(
            Name="ScheduledWorker",
            Cpu=200,
            Memory=300,
            Essential=True,
            Image="<image>",
            EntryPoint=['<entry_point>']
        ),  
    ],  
)

  2. The role to run the task:

    from troposphere import iam

run_task_role = iam.Role(
    "RunTaskRole",
    template=template,
    AssumeRolePolicyDocument=dict(Statement=[dict(
        Effect="Allow",
        Principal=dict(Service=["events.amazonaws.com"]),
        Action=["sts:AssumeRole"],
    )]),
    Path="/",
    Policies=[
        iam.Policy(
            PolicyName="RunTaskPolicy",
            PolicyDocument=dict(
                Statement=[dict(
                    Effect="Allow",
                    Action=[
                        "ecs:RunTask",
                    ],
                    Resource=["*"],
                    Condition=dict(
                        ArnEquals={
                            "ecs:cluster": GetAtt(cluster, "Arn"),
                        }
                    )
                )],
            ),
        ),
    ],
)

  3. The event rule that schedules the task:

    from troposphere.events import Rule, Target, EcsParameters

Rule(
    "SchedulingRule",
    template=template,
    Description="My schedule task rule",
    State="ENABLED",
    ScheduleExpression="rate(30 minutes)",
    Targets=[
        Target(
            Id="ScheduledWorkerTaskDefinitionTarget",
            RoleArn=GetAtt(run_task_role, "Arn"),
            Arn=GetAtt(cluster, "Arn"),
            EcsParameters=EcsParameters(
                TaskCount=1,
                TaskDefinitionArn=Ref(scheduled_worker_task_definition),
            ),
        ),
    ]
)

Where the JSON output looks like:

    ...
    "ScheduledWorkerTask": {
        "Condition": "Deploy",
        "Properties": {
            "ContainerDefinitions": [
                {
                    "Cpu": 200,
                    "EntryPoint": [
                        "<entry_point>"
                    ],
                    "Essential": "true",
                    "Image": {
                        "Fn::Join": [
                            "",
                            [
                                {
                                    "Ref": "AWS::AccountId"
                                },
                                ".dkr.ecr.",
                                {
                                    "Ref": "AWS::Region"
                                },
                                ".amazonaws.com/",
                                {
                                    "Ref": "ApplicationRepository"
                                },
                                ":",
                                "<image>"
                            ]
                        ]
                    },
                    "Memory": 300,
                    "Name": "ScheduledWorker"
                }
            ]
        },
        "Type": "AWS::ECS::TaskDefinition"
    },

    "SchedulingRule": {
        "Properties": {
            "Description": "My schedule task rule",
            "ScheduleExpression": "rate(30 minutes)",
            "State": "ENABLED",
            "Targets": [
                {
                    "Arn": {
                        "Fn::GetAtt": [
                            "Cluster",
                            "Arn"
                        ]
                    },
                    "EcsParameters": {
                        "TaskCount": 1,
                        "TaskDefinitionArn": {
                            "Ref": "ScheduledWorkerTask"
                        }
                    },
                    "Id": "ScheduledWorkerTaskDefinitionTarget",
                    "RoleArn": {
                        "Fn::GetAtt": [
                            "RunTaskRole",
                            "Arn"
                        ]
                    }
                }
            ]
        },
        "Type": "AWS::Events::Rule"
    },

    "RunTaskRole": {
        "Properties": {
            "AssumeRolePolicyDocument": {
                "Statement": [
                    {
                        "Action": [
                            "sts:AssumeRole"
                        ],
                        "Effect": "Allow",
                        "Principal": {
                            "Service": [
                                "events.amazonaws.com"
                            ]
                        }
                    }
                ]
            },
            "Path": "/",
            "Policies": [
                {
                    "PolicyDocument": {
                        "Statement": [
                            {
                                "Action": [
                                    "ecs:RunTask"
                                ],
                                "Condition": {
                                    "ArnEquals": {
                                        "ecs:cluster": {
                                            "Fn::GetAtt": [
                                                "Cluster",
                                                "Arn"
                                            ]
                                        }
                                    }
                                },
                                "Effect": "Allow",
                                "Resource": [
                                    "*"
                                ]
                            }
                        ]
                    },
                    "PolicyName": "RunTaskPolicy"
                }
            ]
        },
        "Type": "AWS::IAM::Role"
    },
    ...

More info at https://jeanphix.github.io/2017/10/04/ecs-scheduled-cron-task-with-cloudformation/

meyerson
  • 4,710
  • 1
  • 20
  • 20
jeanphix
  • 174
  • 1
  • 4
  • good rundown of what needs to happen to trigger tasks - its worth mentioning that you the cluster better have an ec2 ready to run this scheduled task. I believe that this template can be extended to accommodate this pretty easily! – meyerson Oct 23 '17 at 20:32
  • @meyerson How would you go about ensuring this? – Joe Alamo Apr 25 '18 at 15:21
  • 1
    @JoeAlamo - an ec2 autoscaling group associated with your ecs cluster that either keeps a machine on at all time OR keeps it at zero but additionally have service level scaling group that will trigger scale-out events – meyerson Apr 25 '18 at 17:06
  • @meyerson Would an autoscaling approach like http://garbe.io/blog/2017/04/12/a-better-solution-to-ecs-autoscaling/ work for this situation? i.e. have your autoscaling rules and alarms set up so there is always space on your cluster for a new scheduled task to run – Joe Alamo Apr 26 '18 at 09:47
  • 1
    @JoeAlamo that's right - one ec2 autoscaling group that always has room for at least one more task (where you have to decide how big a task is), combined with services that add/remove tasks based on their own, individual scaling rules (serivce scaling). – meyerson Apr 26 '18 at 22:08
0

You can do this with both boto and cloudformation. If you do it with cloudformation, you would need to use a Lambda-backed custom resource. And you can do it with Troposphere as well.

Troposphere Example:

# Read in the code
fname = "lambda_code.py"
try:
  with open(fname) as target:
    code = target.readlines()
except Exception as e:
  print(e)

# Create the custom resource.
CustomResource = template.add_resource(Function(
  "CustomResource",
  Description="Creates and Runs a Lambda function.",
  FunctionName="myFunctionName",
  Code=Code(
    ZipFile=Join("",code)
  ),
  Runtime="python2.7",
  Role=GetAtt("lambaRole","Arn"),
  Handler="index.decide_action",
  MemorySize='128',
  Timeout='10'
))

As for the creating of tasks in boto, check out the Boto3 Documentation.

Additional Resources:

Sathed
  • 836
  • 1
  • 11
  • 21