kube-dns pods does not run when install with kops

Question

I've setup a standalone k8s cluster using kops, so I want all pods running on only 1 master too. The cluster started well but kube-dns and kube-autoscaler pods does not running, it's in pending status. This is node describe

kubectl describe nodes
Name:               ip-10-0-3-184.ap-southeast-1.compute.internal
Roles:              master
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/instance-type=m3.large
                    beta.kubernetes.io/os=linux
                    failure-domain.beta.kubernetes.io/region=ap-southeast-1
                    failure-domain.beta.kubernetes.io/zone=ap-southeast-1a
                    kops.k8s.io/instancegroup=master-ap-southeast-1a
                    kubernetes.io/hostname=ip-10-0-3-184.ap-southeast-1.compute.internal
                    kubernetes.io/role=master
                    node-role.kubernetes.io/master=
Annotations:        node.alpha.kubernetes.io/ttl=0
                    volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:             node-role.kubernetes.io/master:NoSchedule

but /var/log/kube-scheduler.log is:

I0526 12:49:54.629475       1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-autoscaler-787d59df8f-8jgn7
I0526 12:49:54.629570       1 factory.go:1251] Updating pod condition for kube-system/kube-dns-autoscaler-787d59df8f-8jgn7 to (PodScheduled==False)
I0526 12:50:09.706382       1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-7785f4d7dc-rqzdq
I0526 12:50:09.706484       1 factory.go:1251] Updating pod condition for kube-system/kube-dns-7785f4d7dc-rqzdq to (PodScheduled==False)
I0526 12:50:10.632285       1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-autoscaler-787d59df8f-8jgn7
I0526 12:50:10.632371       1 factory.go:1251] Updating pod condition for kube-system/kube-dns-autoscaler-787d59df8f-8jgn7 to (PodScheduled==False)
I0526 12:50:41.709687       1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-7785f4d7dc-rqzdq
I0526 12:50:41.709796       1 factory.go:1251] Updating pod condition for kube-system/kube-dns-7785f4d7dc-rqzdq to (PodScheduled==False)
I0526 12:50:42.635260       1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-autoscaler-787d59df8f-8jgn7
I0526 12:50:42.635361       1 factory.go:1251] Updating pod condition for kube-system/kube-dns-autoscaler-787d59df8f-8jgn7 to (PodScheduled==False)

I can fix it manually by using command kubectl taint nodes --all node-role.kubernetes.io/master- but I want it is enable when I create the cluster with kops create cluster

Thanks

score 0 · Answer 1 · answered May 28 '18 at 14:35

The Taints are set in kops templates for all types of Kubernetes networking add-ons.

It is not possible for now to enable/disable that part of the template with command line options.

But, there is a possibility to configure the taints for an instance group:

kops edit ig master-us-west-1c

Instance Groups
By default, a cluster has:

An instance group called nodes spanning all the zones; these instances are your workers.

One instance group for each master zone, called master- (e.g. master-us-east-1c). These normally have minimum size and maximum size = 1, so they will run a single instance. We do this so that the cloud will always relaunch masters, even if everything is terminated at once. We have an instance group per zone because we need to force the cloud to run an instance in every zone, so we can mount the master volumes - we cannot do that across zones.

Adding Taints or Labels to an Instance Group
If you're running Kubernetes 1.6.0 or later, you can also control taints in the InstanceGroup. The taints property takes a list of strings. The following example would add two taints to an IG, using the same edit -> update -> rolling-update process as above.

Additionally, nodeLabels can be added to an IG in order to take advantage of Pod Affinity. Every node in the IG will be assigned the desired labels. For more information see the labels documentation.

metadata:
  creationTimestamp: "2016-07-10T15:47:14Z"
  name: nodes
spec:
  machineType: m3.medium
  maxSize: 3
  minSize: 3
  role: Node
  taints:
  - dedicated=gpu:NoSchedule
  - team=search:PreferNoSchedule
  nodeLabels:
    spot: "false"

This is not my case, I have an instance group with only 1 master and 0 node, so I want to run pods on the master. This lead the above problem. But never mind, I solved this by create a custom user-data script to untaint the master to allow it allocatable pods — Jin, May 29 '18 at 09:10

kube-dns pods does not run when install with kops

1 Answers1