3

I ran into this issue earlier whereby I already have a working cluster with one master node and one worker node including the Calico CNI installed. I wanted to have another worker node, so what I did was; I cloned the existing worker node, change the IP address, giving it its own hostname and added the IP address to the master node's /etc/hosts. I understand that this is not the right way to do it but me and my colleagues is trying this out for fun. Anyway, I've added the second worker node (previously cloned from first worker node) to the cluster using kubeadm join. Then, I realized that no Calico pod is being created or scheduled on that worker node. Also, the other Calico pod also seems to be failing, its showing 0/1 when I do kubectl get pods.

Can anyone explain to me why does the Calico pod not scheduled on the second worker node? From my understanding, a DaemonSet schedules a copy of the Pods on each of the nodes, but will it schedule a pod on a newly joined worker node?

P.S. I've already fix the Calico pod issue. What I did was remove Calico completely and added it back to the cluster.

Joe S
  • 83
  • 1
  • 1
  • 11
  • 1
    Could you check the node status, or if there are any taints attached to the second worker node, by doing `kubectl describe <>`. And also check the events related to the respective `calico` pod why it got stuck in pending state, add it here as it is slightly difficult to know what's actually happening. – BinaryMonster Sep 23 '20 at 07:06
  • @BinaryMonster the second worker node is showing `Taints: `. The event of the `calico` pod is showing the error similar to this: https://github.com/projectcalico/calico/issues/2834 `calico/node is not ready: BIRD is not ready: Error querying BIRD`. I got this error too when I was setting up a new cluster few weeks ago, but I did solve it by using the same solution found in the link. – Joe S Sep 23 '20 at 07:24
  • Looks like a tough one and interesting, as this is a cloned copy or vm image I guess there might be some places where the node still has ip address related to old node or something related to network configurations. Once check `ip r` if the ip addresses are new as defined or something happening over there. However, this is kinda interesting thing you tried using vm image to bring in a worker in quick time. – BinaryMonster Sep 23 '20 at 08:30

1 Answers1

0

I also met a similar scene that my calico-node even not scheduled to other nodes

My ENV: calico image 3.20.3 only one master on-premise(1.20.12) node and everything runs on it;

one day I got two new machines and use kubeadm join to add those two nodes; it threw me something like

21870 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
2月 15 21:42:07 work1 kubelet[21870]: E0215 21:42:07.243451   21870 file_linux.go:60] Unable to read config path "/etc/kubernetes/manifests": path does not exist, ignoring
2月 15 21:42:08 work1 kubelet[21870]: E0215 21:42:08.243625   21870 file_linux.go:60] Unable to read config path "/etc/kubernetes/manifests": path does not exist, ignoring
2月 15 21:42:09 work1 kubelet[21870]: E0215 21:42:09.243799   21870 file_linux.go:60] Unable to read config path "/etc/kubernetes/manifests": path does not exist, ignoring
2月 15 21:42:10 work1 kubelet[21870]: E0215 21:42:10.243931   21870 file_linux.go:60] Unable to read config path "/etc/kubernetes/manifests": path does not exist, ignoring
2月 15 21:42:10 work1 kubelet[21870]: W0215 21:42:10.747428   21870 cni.go:239] Unable to update cni config: no networks found in /etc/cni/net.d

What I did is to copy the dir /etc/cni/net.d and /opt/cni/bin from master node to the destination node;

Cuz found in calico.yaml (manifest way), it mount two hostPath that's used to install CNI:

        # Used to install CNI.
        - name: cni-bin-dir
          hostPath:
            path: /opt/cni/bin
        - name: cni-net-dir
          hostPath:
            path: /etc/cni/net.d

Then everything goes well;

But I have no idea why it didn't work before;

Hua Mai
  • 99
  • 9