Microk8s pods are stuck on pending on Ubuntu

Question

I already looked for solutions in the internet but could not find anything useful. I have a server hostet, which runs Ubuntu 18.04. The Pods are always at the pending state and do not want to run.

NAMESPACE            NAME                                                  READY   STATUS    RESTARTS   AGE
container-registry   pod/registry-7cf58dcdcc-hf8gx                         0/1     Pending   0          12m
kube-system          pod/coredns-588fd544bf-4m6mj                          0/1     Pending   0          12m
kube-system          pod/dashboard-metrics-scraper-db65b9c6f-gj5x4         0/1     Pending   0          12m
kube-system          pod/heapster-v1.5.2-58fdbb6f4d-q6plc                  0/4     Pending   0          12m
kube-system          pod/hostpath-provisioner-75fdc8fccd-6mdvc             0/1     Pending   0          12m
kube-system          pod/kubernetes-dashboard-67765b55f5-8xsh5             0/1     Pending   0          12m
kube-system          pod/monitoring-influxdb-grafana-v4-6dc675bf8c-82fg4   0/2     Pending   0          12m

When I enter mickrok8s.kubectl describe nodes I am getting this output...

 Normal   Starting                 15s    kubelet, h2860142.stratoserver.net     Starting kubelet.
  Warning  InvalidDiskCapacity      15s    kubelet, h2860142.stratoserver.net     invalid capacity 0 on image filesystem
  Normal   NodeHasSufficientPID     15s    kubelet, h2860142.stratoserver.net     Node h2860142.stratoserver.net status is now: NodeHasSufficientPID
  Normal   NodeHasNoDiskPressure    15s    kubelet, h2860142.stratoserver.net     Node h2860142.stratoserver.net status is now: NodeHasNoDiskPressure
  Normal   NodeHasSufficientMemory  15s    kubelet, h2860142.stratoserver.net     Node h2860142.stratoserver.net status is now: NodeHasSufficientMemory
  Normal   NodeHasSufficientPID     3s     kubelet, h2860142.stratoserver.net     Node h2860142.stratoserver.net status is now: NodeHasSufficientPID

The output of "microk8s.kubectl get nodes -o wide"...

NAME                        STATUS     ROLES    AGE   VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
h2860142.stratoserver.net   NotReady   <none>   23h   v1.18.3   85.214.165.16   <none>        Ubuntu 18.04.4 LTS   4.15.0           containerd://1.2.5

The output of microk8s.kubectl get pods -A

NAMESPACE            NAME                                              READY   STATUS    RESTARTS   AGE
container-registry   registry-7cf58dcdcc-hf8gx                         0/1     Pending   0          63m
kube-system          coredns-588fd544bf-4m6mj                          0/1     Pending   0          63m
kube-system          dashboard-metrics-scraper-db65b9c6f-gj5x4         0/1     Pending   0          63m
kube-system          heapster-v1.5.2-58fdbb6f4d-q6plc                  0/4     Pending   0          63m
kube-system          hostpath-provisioner-75fdc8fccd-6mdvc             0/1     Pending   0          63m
kube-system          kubernetes-dashboard-67765b55f5-8xsh5             0/1     Pending   0          63m
kube-system          monitoring-influxdb-grafana-v4-6dc675bf8c-82fg4   0/2     Pending   0          63m

Output of "microk8s.inspect"

Inspecting Certificates                                                                
Inspecting services                                                                    
  Service snap.microk8s.daemon-cluster-agent is running                                
  Service snap.microk8s.daemon-flanneld is running                                     
  Service snap.microk8s.daemon-containerd is running                                   
  Service snap.microk8s.daemon-apiserver is running                                    
  Service snap.microk8s.daemon-apiserver-kicker is running                             
  Service snap.microk8s.daemon-proxy is running                                        
  Service snap.microk8s.daemon-kubelet is running                                      
  Service snap.microk8s.daemon-scheduler is running                                    
  Service snap.microk8s.daemon-controller-manager is running                           
  Service snap.microk8s.daemon-etcd is running                                         
  Copy service arguments to the final report tarball                                   
Inspecting AppArmor configuration                                                      
Gathering system information                                                           
  Copy processes list to the final report tarball                                      
  Copy snap list to the final report tarball                                           
  Copy VM name (or none) to the final report tarball                                   
  Copy disk usage information to the final report tarball                              
  Copy memory usage information to the final report tarball                            
  Copy server uptime to the final report tarball                                       
  Copy current linux distribution to the final report tarball                          
  Copy openSSL information to the final report tarball                                 
  Copy network configuration to the final report tarball                               
Inspecting kubernetes cluster                                                          
  Inspect kubernetes cluster                                                           

Building the report tarball                                                            
  Report tarball is at /var/snap/microk8s/1422/inspection-report-20200609_120308.tar.gz

So how can I solve this problem?

Thank you :)

share output of mickrok8s.kubectl get nodes -o wide and mickrok8s.kubectl get events — Arghya Sadhu, Jun 09 '20 at 09:16
"get events" has the same output of "describe nodes". Above the output of "get nodes -o wide" — volkanb, Jun 09 '20 at 09:46
How do I get the logs? The outputs of any logs-statement is a Error with not found. — volkanb, Jun 09 '20 at 10:27
run this command journalctl or journalctl -u microk8s.daemon-kubelet — Arghya Sadhu, Jun 09 '20 at 10:32
have you enabled `storage`? you can check this with `microk8s.status` — Griffin, Jun 14 '20 at 23:48
Please following instructions on here: https://containersolutions.github.io/runbooks/posts/kubernetes/pod-stuck-in-pending-status/ and update with where you get stuck or can't progress. — ianmiell, Jun 15 '20 at 12:29
Micrk8s is running on a hosted server where my website is hosted. Could the issue be something with ports? — volkanb, Jun 25 '20 at 14:46

score 0 · Answer 1 · answered May 24 '21 at 19:24

There's a section in the docs about this.

If a Pod is stuck in Pending it means that it can not be scheduled onto a node. Generally this is because there are insufficient resources of one type or another that prevent scheduling. Look at the output of the kubectl describe ...

In my case kubectl describe pods <podname> said:

Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  6m1s (x3 over 7m6s)  default-scheduler  0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.

Searching for that warning gave me what I needed. With luck, a useful warning might show up there for you too.

score 0 · Answer 2 · answered Oct 18 '21 at 13:43

You might have cgroups disabled in your ubuntu installation. If you are running on a Raspberry Pi for example cgroups are disabled by default.

cgroups are required for microk8s to work.

You might want to enable cgroups as described here:

invalid capacity 0 on image filesystem - Not ZFS

Microk8s pods are stuck on pending on Ubuntu

2 Answers2