6

I tried this doc to install and setup Kubernetes in Ubuntu VM. I have finished upto 3/4 and now kube-dns pod is in pending status. How can i figure out this? here is the result for kubectl get pods --namespace=kube-system and kubectl describe pod <pod name>

# kubectl get pods --namespace=kube-system
NAME                              READY     STATUS    RESTARTS   AGE
dummy-2088944543-jk2t2            1/1       Running   0          3h
etcd-ubuntu                       1/1       Running   0          3h
kube-apiserver-ubuntu             1/1       Running   0          3h
kube-controller-manager-ubuntu    1/1       Running   0          3h
kube-discovery-1769846148-h88v4   1/1       Running   0          3h
kube-dns-2924299975-dfp17         0/4       Pending   0          3h
kube-proxy-zdcxw                  1/1       Running   0          3h
kube-scheduler-ubuntu             1/1       Running   0          3h
weave-net-xwfhj                   2/2       Running   0          2h

# kubectl describe pod kube-dns-2924299975-dfp17
Error from server (NotFound): pods "kube-dns-2924299975-dfp17" not found
Lakmal Vithanage
  • 2,767
  • 7
  • 42
  • 58
  • Try using kubectl command to inspect the labels in the configuration data of the pod. Kubernetes well only schedule it only a host/kubelet which has matching labels. It could be that the pod for example has labels stating to only run in on a node which has a label "region=infrastructure" or something similar (that's just a made up example).also each kubelet has a notion of whether it is "schedulable" and it won't get anything automatically deployed onto it if this is not turned on. The last idea is that it possibly cannot download the container image. You can check the logs for that – Chunko Feb 14 '17 at 09:51
  • @lakmal-vithanage can you post the output of `kubectl describe pod `? – Antoine Cotten Feb 14 '17 at 10:37
  • @AntoineCotten, I have updated the question – Lakmal Vithanage Feb 14 '17 at 11:28
  • @lakmal-vithanage you forgot `--namespace=kube-system` in the command, hence the error. Could you update the question one more time? – Antoine Cotten Feb 14 '17 at 12:57

1 Answers1

16

Cause

Most likely a lack of available computing resources in your cluster.

If you're using the example in cluster/addons/dns you're certainly using a Deployment with resource requests, highlighted if you click the link. It could be that your other pods are already requesting all the available resources in the cluster, therefore your pod doesn't get scheduled.

You can confirm that theory with kubectl --namespace=kube-system describe pod kube-dns-2924299975-dfp17 and look for the following event:

Reason                Message
------                -------
FailedScheduling      pod (kube-dns-2924299975-dfp17) failed to fit in any node
fit failure summary on nodes : Insufficient cpu (3)

You can also describe your nodes with kubectl describe node <node-name> and look at the last information:

Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.
  CPU Requests  CPU Limits      Memory Requests Memory Limits
  ------------  ----------      --------------- -------------
  320m (8%)     300m (7%)       150Mi (1%)      150Mi (1%)

In your case either the CPU or memory allocation should be close to 100%.

Solution

  • Add more computing resources / nodes to your cluster (preferred)
  • Remove the resource requests from your pod(s), at the risk of overcommitting your resources
Antoine Cotten
  • 2,673
  • 18
  • 37