6

I created an AWS EKS Cluster with the terraform-aws-eks module. Terraform version is 1.0.6, aws provider version is 3.60.0. With these versions i should be able to use aws_autoscaling_group_tag resource to tag the ASGs which are created by EKS.

My problem is that nodegroups in the module is a map of maps (described here) and i don't know how to iterate over my nodegroups to tag all ASGs within them. Here is the example from terraform:

resource "aws_eks_node_group" "example" {
  cluster_name    = "example"
  node_group_name = "example"

  # ... other configuration ...
}

resource "aws_autoscaling_group_tag" "example" {
  for_each = toset(
    [for asg in flatten(
      [for resources in aws_eks_node_group.example.resources : resources.autoscaling_groups]
    ) : asg.name]
  )

  autoscaling_group_name = each.value

  tag {
    key   = "k8s.io/cluster-autoscaler/node-template/label/eks.amazonaws.com/capacityType"
    value = "SPOT"

    propagate_at_launch = false
  }
}

In this case there is one specific nodegroup. But in my case there are 3 nodegroups and i want all ASGs to be tagged. I haven't worked much with loops in terraform so far and i am even not sure if it will work. I appreciate any help!

volwei
  • 61
  • 3
  • Why not have the module tag the node group themselves with [`additional_tags`](https://github.com/terraform-aws-modules/terraform-aws-eks/blob/253f927465ccac36bcadefcfb1fd0e39b6244d73/modules/node_groups/launch_template.tf#L95-L96)? – ydaetskcoR Sep 30 '21 at 15:11
  • I don't know if additional_tags also tags the autoscaling groups. I'll try. Thanks. – volwei Oct 11 '21 at 12:26
  • If you are open for option; `eksctl` has the label automatically [propagate](https://eksctl.io/usage/autoscaling/) for you, and it works for both self-managed and managed node group. Currently, terraform-aws-eks module support label propagation for managed node group, self-managed node group is not supported yet. – gohm'c May 12 '22 at 15:00

1 Answers1

1

You mentioned you use terraform-aws-eks module. When you use the module, the definition of the node groups (managed or self-managed) is part of this module. This module basically uses a submodule eks-managed-node-group and this submodule supports tagging. The module creates Launch templates and ASG groups based on those templates.

You don't need to tag the ASG outside the module. You better not do it, since it will drift your infrastructure configurations: the next time you would run apply on the terraform-aws-eks module code, the tags might be changed.

Your configuration should look something like this:

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 18.0"
  ..
  ..

  # EKS Managed Node Group(s)
  eks_managed_node_group_defaults = {
    tags = {
      "k8s.io/cluster-autoscaler/node-template/label/eks.amazonaws.com/capacityType" = "SPOT"
    }
  }

  eks_managed_node_groups = {
    blue = {}
    green = {
      min_size     = 1
      max_size     = 10
      desired_size = 1

      instance_types = ["t3.large"]
    }
  }

Specifying the tags at eks_managed_node_group_defaults will apply to all node groups. You can config it per node group (in the above example, blue and green) if you want them to be different.

Note that using tags means all the created resources would be tagged. I don't find it problematic, but if you do then a different solution is needed here.

Here is a link to all supported inputs for the submodule.

You can check the modules documentations for more examples.

If I understand you correctly, you want to create ASG group with the capacity type of SPOT instances. The terraform-aws-eks module has the capacity_type parameter for that which is described as Type of capacity associated with the EKS Node Group. Valid values: ON_DEMAND, SPOT

Then, it's much simpler to achieve what you look for:

  eks_managed_node_group_defaults = {
    capacity_type  = "SPOT"
  }
Chen A.
  • 10,140
  • 3
  • 42
  • 61