3

I have deployed the Kubernetes dashboard which ended up in CrashLoopBackOff status. When I run:

$ kubectl logs kubernetes-dashboard-767dc7d4d-mc2sm --namespace=kube-system

the output is:

Error from server: Get https://10.4.211.53:10250/containerLogs/kube-system/kubernetes-dashboard-767dc7d4d-mc2sm/kubernetes-dashboard: dial tcp 10.4.211.53:10250: connect: no route to host

How can I fix this? Does this means that the port 10250 isn't open?

Update：

@LucaBrasi
Error from server (NotFound): pods "kubernetes-dashboard-767dc7d4d-mc2sm" not found

systemctl status kubelet --full Output is ： 

kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since 一 2018-09-10 15:04:57 CST; 1 day 23h ago
     Docs: https://kubernetes.io/docs/
 Main PID: 93440 (kubelet)
    Tasks: 21
   Memory: 78.9M
   CGroup: /system.slice/kubelet.service
           └─93440 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni

Output for kubectl get pods --all-namespaces

NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-78fcdf6894-qh6zb 1/1 Running 2 3d kube-system coredns-78fcdf6894-xbzgn 1/1 Running 1 3d kube-system etcd-twsr-whtestserver01.garenanet.com 1/1 Running 2 3d kube-system kube-apiserver-twsr-whtestserver01.garenanet.com 1/1 Running 2 3d kube-system kube-controller-manager-twsr-whtestserver01.garenanet.com 1/1 Running 2 3d kube-system kube-flannel-ds-amd64-2bnmx 1/1 Running 3 3d kube-system kube-flannel-ds-amd64-r58j6 1/1 Running 0 3d kube-system kube-flannel-ds-amd64-wq6ls 1/1 Running 0 3d kube-system kube-proxy-ds7lg 1/1 Running 0 3d kube-system kube-proxy-fx46d 1/1 Running 0 3d kube-system kube-proxy-ph7qq 1/1 Running 2 3d kube-system kube-scheduler-twsr-whtestserver01.garenanet.com 1/1 Running 1 3d kube-system kubernetes-dashboard-767dc7d4d-mc2sm 0/1 CrashLoopBackOff 877 3d

Rukeith
  • 665
  • 1
  • 8
  • 22
  • Port 10250 is typically the kubelet agent port. Can you show the output of: kubectl describe pod kubernetes-dashboard-767dc7d4d-mc2sm and systemctl status kubelet --full – Luca Brasi Sep 12 '18 at 05:31
  • It means that something is not right with your networking/CNI configs. What overlay did you install? – Rico Sep 12 '18 at 05:33
  • @Rico My network provider is flannel – Rukeith Sep 12 '18 at 06:18
  • @Rukeith how did you deploy the cluster (what tool) and did you deploy kubernetes dashboard from the [github](https://github.com/kubernetes/dashboard)? Also the flannel was deployed manually or you used automatic deployment, for example (CoreOS's Tectonic)? – aurelius Sep 12 '18 at 10:51
  • @aurelius I install flannel with this tutorial https://www.techrepublic.com/article/how-to-install-a-kubernetes-cluster-on-centos-7/ I deploy dashboard with these yml file. https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml – Rukeith Sep 12 '18 at 13:10
  • Can you run kubectl get pods - -all-namespaces see if all pods are running – Yonah Dissen Sep 13 '18 at 04:21

1 Answers1

2

I had the same issue when I reproduced all the steps from the tutorial you've linked - my dashboard was in CrashLoopBackOffstate. After I performed this steps and applied new dashboard yaml from the official github documentation (there seems to be no difference from the one you've posted), the dashboard was working properly.

First, list all the objects related to Kubernetes dashboard:

kubectl get secret,sa,role,rolebinding,services,deployments --namespace=kube-system | grep dashboard

Delete them:

kubectl delete deployment kubernetes-dashboard --namespace=kube-system 
kubectl delete service kubernetes-dashboard  --namespace=kube-system 
kubectl delete role kubernetes-dashboard-minimal --namespace=kube-system 
kubectl delete rolebinding kubernetes-dashboard-minimal --namespace=kube-system
kubectl delete sa kubernetes-dashboard --namespace=kube-system 
kubectl delete secret kubernetes-dashboard-certs --namespace=kube-system
kubectl delete secret kubernetes-dashboard-key-holder --namespace=kube-system

Now apply Kubernetes dashboard yaml:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

Please tell me if this worked for you as well, and if it did, treat it as a workaround as I don't know the reason yet - I am investigating.

Cyclic3
  • 149
  • 2
  • 10
aurelius
  • 3,433
  • 1
  • 13
  • 22