5

I have the following Terraform code to update a service with a new task definition:

resource "aws_ecs_task_definition" "app_definition" {
  family = "my-family"

  container_definitions = "${data.template_file.task_definition.rendered}"
  network_mode          = "bridge"
}

resource "aws_ecs_service" "app_service" {
  name            = "my-service"
  cluster         = "my-cluster"
  task_definition = "${aws_ecs_task_definition.app_definition.arn}"
  desired_count   = "1"
  iam_role        = "my-iam-role"
}

When updating my service, the last revision of my task definition becomes inactive. As a result, I can not select it when trying to manually roll back to a previous revision in the ECS console:

Error: No active task definition found

Ideally, I want to keep the last X revisions active so I can always manually roll back via the console if something goes wrong.

How can I achieve that?

ydaetskcoR
  • 53,225
  • 8
  • 158
  • 177
bitbrain
  • 603
  • 5
  • 12

2 Answers2

9

Terraform doesn't currently allow for this and its resource lifecycle model means that when you replace something (task definitions are immutable) Terraform must create a new one and destroy the old one.

With ECS task definitions also can't really be destroyed and instead are just marked as inactive as there may be tasks currently deployed that are using it until they are updated by the service to the new task definition.

There's 2 common ways of dealing with this and the need to be able to roll back to a previous version of a task definition.

The first is simply not to use Terraform to manage the task definition beyond initial creation and use something like the AWS ECS CLI tool to do this instead.

The other option, and the one that I use, is to have my CI (Gitlab CI in our case) generate a Docker image tagged with the commit SHA of the application to be deployed and then Terraform updates the task definition to the new commit SHA tagged image on an apply as well as updating the ECS service with the new task definition ARN.

When we want to roll back we use our CI's ability to roll back to a different commit, launching just the deploy job with the old commit SHA and so deploying the old image.

This keeps Terraform pretty agnostic of what's being deployed and makes the CI system responsible for deploying the required version which is normally latest but sometimes a specific commit if we have a manual click to deploy and of course the target previous version when rolling back.

It does mean that you can't launch roll backs through the AWS console but I actually like this as I want the CI system to be the source of truth for what is deployed at any time.

ydaetskcoR
  • 53,225
  • 8
  • 158
  • 177
-4

A very simple approach is to hook into the lifecycle of Terraform:

resource "aws_ecs_task_definition" "app_definition" {
  family = "my-family"

  container_definitions = "${data.template_file.task_definition.rendered}"
  network_mode          = "bridge"

  # make sure Terraform does not unregister the task definition
  lifecycle {
    prevent_destroy = true
  }
}

As discussed in this Pull Request it prevents the destruction of the old task definition, thus keeps all task definitions active.

bitbrain
  • 603
  • 5
  • 12
  • did you even try that yourself? It doesnt work: `* aws_ecs_task_definition.mydef: aws_ecs_task_definition.mydef: the plan would destroy this resource, but it currently has lifecycle.prevent_destroy set to true. To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or adjust the scope of the plan using the -target flag.` – red888 Dec 28 '18 at 12:28
  • @red888 I discovered that this solution works but not always - in the beginning I ran without any issues. At some point I got the same error. I am curious what is causing this! – bitbrain Jan 03 '19 at 15:18
  • i doubt this approach has ever worked except the first time you run it – Manuel G Jan 13 '23 at 10:59