Why creating EKS cluster auto-creates security groups I already have in place

Question

I am trying to create an EKS cluster via CloudFormation. I have read all the EKS Security Group guidelines by Amazon and already put in place my security groups as I want clearer more tidy naming and also to be able to define the intricacies between these and some others (BastionHost SG and RDS SG) beforehand.

For the love of God I cannot understand why it keeps creating the Cluster Security Group by itself ignoring the one that I am passing as reference in my template and also the same thing kind of happens in the NodeGroup's remote access security group where I specify my Bastion Host's security group. Instead of accepting it it goes on to create a new security group of its own which has as source the security group of my BastionHost.

Literally confused. Can I overcome this?

Update: So I am having the 3 security groups that Amazon suggests for my EKS. Let's call them cluster-sg, control-plane-sg, and nodegroup-sg. Also assume that they have been configured as per the guide above adopting the "recommended" inbound/outbound traffic guidelines and not the minimum (although I don't see this playing an important role at this part). Additionally there is the security group of a separate EC2 instance which is my Bastion Host, let's call it bastion-sg.

My CloudFormation template looks like this:

EKSCluster:
Type: 'AWS::EKS::Cluster'
Properties:
  Name: 'my-cluster'
  Version: '1.17'
  ResourcesVpcConfig:
    SecurityGroupIds:
      - !Ref clusterSG #do I need this cluster-sg here? do I need also nodegroup-sg? do I need both?
    SubnetIds:
      - !Ref PrivateSubnet1
      - !Ref PrivateSubnet2
      - !Ref PrivateSubnet3
  RoleArn: !GetAtt ClusterInstanceRole.Arn

NodeGroupCluster:
Type: AWS::EKS::Nodegroup
Properties:
  ClusterName: !Ref EKSCluster
  DiskSize: !Ref ClusterDiskSize
  InstanceTypes: !Ref NodeInstanceTypes
  ForceUpdateEnabled: false
  NodegroupName: 'cluster-nodegroup'
  NodeRole: !GetAtt NodeInstanceRole.Arn #this is a resource that I haven't provided
  RemoteAccess:
   Ec2SshKey: !Ref EC2KeyPair
   SourceSecurityGroups:
    - !Ref bastionSG 
  ScalingConfig:
    DesiredSize: !Ref DesiredNodeSize
    MaxSize: !Ref MaximumNodeSize
    MinSize: !Ref MinimumNodeSize
  Subnets:
    - !Ref PrivateSubnet1
    - !Ref PrivateSubnet2
    - !Ref PrivateSubnet3

All in all the issue is two-fold: a) I seem to be missing where to put what in terms of configuration and security groups. i.e. I have 3 security groups recommended by Amazon for the whole cluster but only two places where security groups are accepted. b) Any combination that I have tried (as per my original post) does not take into consideration the cluster-sg but auto-creates one on its own which is not convenient for my IaaC and auto-deploy philosophy.

What do the relevant sections of your cloudformation template look like? — jordanm, Feb 09 '21 at 16:37
@jordanm I have updated my original post with additional information and code. Thank you! — George S., Feb 10 '21 at 09:59

score 4 · Accepted Answer · answered Mar 28 '21 at 14:49

I was going through the same issue. The EKS Control Plane is on a separate VPC somewhere within AWS. When we create the EKS Cluster,it automatically will create an ENI with which it will attach a security group which will be used in communication between control plane and worker nodes. This SG for the cluster will always get created no matter what. Now, if you are passing other security groups as the argument for SG in your cloudformation script it will simply attach those security groups as well with the ENI which was created. If you launch a Nodegroup NG-1 and attach it with a different security group, then make sure to provide the SG in the cluster cloudformation script so that it can attach that security group with the ENI and open respective ports so worker nodes can communicate with the cluster.

I created launchtemplate with a dedicated SG and used the same launchtemplate to create an NodeGroup. Also I mentioned the SG of my Launchtemplate in the cluster script in CF so as to make sure that my nodes can communicate with the control plane else the nodegroups will fail to launch.

So in short just remember following points:

-There will always be a security group created by cluster

-ClusterSG or any other SG which you are passing in your cf cluster script will be attached to the ENI created by EKS on launch.

Please refer to the following documentation to get a better understanding.

https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html

Thank you. Remembering that the Kubernetes Control Plane runs in a dedicated VPC solves a lot of misunderstandings. — Ahmed Gad, Apr 04 '23 at 17:27

Why creating EKS cluster auto-creates security groups I already have in place

1 Answers1