I am trying to create an EKS cluster with cluster.yaml deployment.yaml and service.yaml but nodeSelector is not working as expected

Question

I am trying to create an EKS cluster which has Jenkins running on Jenkins nodes and Nexus running on Nexus nodes for this I am trying to use nodeSelector which is not working as expected I don't understand which part I am missing.

My cluster.yaml for creating the EKS cluster is as follows:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: Devops-Test
  region: ap-south-1

vpc:
  id: vpc-xxxxxx
  cidr: "192.168.0.0/16"
  subnets:
    public:
      ap-south-1a:
        id: subnet-xxxx
      ap-south-1b:
        id: subnet-xxxx
    private:
      ap-south-1a:
        id: subnet-xxxx
      ap-south-1b:
        id: subnet-xxxx

nodeGroups:
  - name: jenkins-public-node-group
    tags: { role: "jenkins" }
    instanceType: t2.medium
    desiredCapacity: 2
  - name: jenkins-private-node-group
    tags: { role: "jenkins" }
    instanceType: t2.medium
    desiredCapacity: 2
    privateNetworking: true
  - name: nexus-public-node-group
    tags: { role: "nexus" }
    instanceType: t2.medium
    desiredCapacity: 2
  - name: nexus-private-node-group
    tags: { role: "nexus" }
    instanceType: t2.medium
    desiredCapacity: 2
    privateNetworking: true

My deployment.yaml is as follows

apiVersion: apps/v1
kind: Deployment
metadata:
  name: devops-tools
  namespace: devops
spec:
  replicas: 2
  selector:
    matchLabels:
      role: jenkins 
  template:
    metadata:
      labels:
        role: jenkins
    spec:
      nodeSelector:
        role: jenkins
      containers:
        - name: jenkins
          image: jenkins:2.60.3
          ports:
            - containerPort: 8080

Finally my service.yaml is as follows

apiVersion: v1
kind: Service
metadata:
  name: jenkins-service
  namespace: devops
spec:
  type: NodePort
  selector:
    role: jenkins
  ports:
    - nodePort: 31429
      port: 8080
      targetPort: 8080

I am expecting Jenkins to run only on nodes tagged with role:jenkins but it is also running on nodes without that tag I have even tried applying label with

kubectl label nodes role=jenkins

and then applying deployment.yaml but deployment still happens on nodes without that label.

score 0 · Accepted Answer · answered Dec 30 '22 at 15:18

You should use labels instead of tags in your cluster.yaml file.
See these docs for more information.

Tags apply to AWS tags, which are irrelevant to Kubernetes. Only labels are relevant when trying to apply node selectors.

BTW - you should also make sure that your node selectors are applied as prescribed to your pods - since pods shouldn't be allowed on nodes without the specified label. From the behavior you are describing - it seems like the pods are being created without a node selector at all.

I am trying to create an EKS cluster with cluster.yaml deployment.yaml and service.yaml but nodeSelector is not working as expected

1 Answers1