1

Problem: Created eks 1.18 cluster with terraform 0.13.5 but spot instances are not joined to the cluster.

➜ k get nodes      
No resources found in default namespace.

➜ terraform -version                               
Terraform v0.13.5
+ provider registry.terraform.io/hashicorp/aws v3.22.0
+ provider registry.terraform.io/hashicorp/kubernetes v1.13.3
+ provider registry.terraform.io/hashicorp/local v2.0.0
+ provider registry.terraform.io/hashicorp/null v3.0.0
+ provider registry.terraform.io/hashicorp/random v2.3.1
+ provider registry.terraform.io/hashicorp/template v2.2.0
module "eks_cluster" {
  source          = "terraform-aws-modules/eks/aws"
  cluster_name    = module.eks_label.id
  cluster_version = "1.18"
  subnets         = var.subnets
  vpc_id          = var.vpc_id

  worker_groups_launch_template = [
    {
      name                                 = "python-spot-pool"
      override_instance_types              = ["m5.xlarge", "m5a.xlarge", "m5zn.xlarge"]
      spot_instance_pools                  = 5
      asg_max_size                         = 5
      asg_desired_capacity                 = 2
      root_volume_size                     = 150
      root_encrypted                       = true
      metadata_http_put_response_hop_limit = 10
      kubelet_extra_args                   = "--node-labels=node.kubernetes.io/lifecycle=spot"
      public_ip                            = false
    }
  ]
  tags = merge(var.tags, {
    created-by = var.created_by
  })
}

So the spot instances created, but they're not joined into the AWS EKS cluster.

➜ aws ec2 describe-instances  G eks
||||  Arn    |  arn:aws:iam::xxx:instance-profile/team-shared-eks2021010923363674750000000a                                               ||||
|||||  GroupName                  |  team-shared-eks20210109232249263900000003                                                                    |||||
||||  GroupName                  |  team-shared-eks20210109232249263900000003                                                                      ||||
||||  kubernetes.io/cluster/team-shared-eks               |  owned                                                                                 ||||
||||  Name                                                    |  team-shared-eks-python-spot-pool-eks_asg                                          ||||
||||  aws:autoscaling:groupName                               |  team-shared-eks-python-spot-pool20210109234259660100000003                        ||||
||||  Arn    |  arn:aws:iam::xxx:instance-profile/team-shared-eks2021010923363674750000000a                                               ||||
|||||  GroupName                  |  team-shared-eks20210109232249263900000003                                                                    |||||
||||  GroupName                  |  team-shared-eks20210109232249263900000003                                                                      ||||
||||  aws:autoscaling:groupName                               |  team-shared-eks-python-spot-pool20210109234259660100000003                        ||||
||||  Name                                                    |  team-shared-eks-python-spot-pool-eks_asg                                          ||||
||||  kubernetes.io/cluster/team-shared-eks               |  owned  

If I replace

worker_groups_launch_template = [
    {
      name                 = "on-demand-pool"
      instance_type        = "t3.small"
      asg_max_size         = 1
      asg_min_size         = 1
      asg_desired_capacity = 1
      root_volume_size     = 150
      kubelet_extra_args   = "--node-labels=workload=app,tech_stack=php"
      public_ip            = false
    },
    {
      name                    = "python-spot-pool"
      override_instance_types = ["m5.xlarge", "m5a.xlarge", "m5zn.xlarge"]
      spot_instance_pools     = 5
      asg_max_size            = 5
      asg_desired_capacity    = 2
      root_volume_size        = 150
      kubelet_extra_args      = "--node-labels=node.kubernetes.io/lifecycle=spot"
      public_ip               = false
    },
  ]

then I get only a single t3.small node in the eks cluster, but again - no spot, however they're present in the AWS account

DmitrySemenov
  • 9,204
  • 15
  • 76
  • 121
  • Perhaps better create an issue at Terraforms site ? – Marged Jan 10 '21 at 01:36
  • Does the ASG get created as expected? If so it's not a specifically Terraform issue but it might give you something to debug. After that you should check kubelet logs on the instances in the ASG. – ydaetskcoR Jan 10 '21 at 12:22

0 Answers0