Kubernetes worker nodes not automatically being assigned podCidr on kubeadm join

Question

I have a multi-master Kubernetes cluster set up, with one worker node. I set up the cluster with kubeadm. On kubeadm init, I passed the -pod-network-cidr=10.244.0.0/16 (using Flannel as the network overlay).

When using kubeadm join on the first worker node, everything worked properly. For some reason when trying to add more workers, none of the nodes are automatically assigned a podCidr.

I used this document to manually patch each worker node, using the kubectl patch node <NODE_NAME> -p '{"spec":{"podCIDR":"<SUBNET>"}}' command and things work fine.

But this is not ideal, I am wondering how I can fix my setup so that just adding the kubeadm join command will automatically assign the podCidr.

Any help would be greatly appreciated. Thanks!

Edit:

I1003 23:08:55.920623       1 main.go:475] Determining IP address of default interface

I1003 23:08:55.920896       1 main.go:488] Using interface with name eth0 and address 

I1003 23:08:55.920915       1 main.go:505] Defaulting external address to interface address ()

I1003 23:08:55.941287       1 kube.go:131] Waiting 10m0s for node controller to sync

I1003 23:08:55.942785       1 kube.go:294] Starting kube subnet manager

I1003 23:08:56.943187       1 kube.go:138] Node controller sync successful

I1003 23:08:56.943212       1 main.go:235] Created subnet manager:

Kubernetes Subnet Manager - kubernetes-worker-06

I1003 23:08:56.943219       1 main.go:238] Installing signal handlers

I1003 23:08:56.943273       1 main.go:353] Found network config - Backend type: vxlan

I1003 23:08:56.943319       1 vxlan.go:120] VXLAN config: VNI=1 Port=0 GBP=false DirectRouting=false

E1003 23:08:56.943497       1 main.go:280] Error registering network: failed to acquire lease: node "kube-worker-02" pod cidr not assigned

I1003 23:08:56.943513       1 main.go:333] Stopping shutdownHandler...

Do you see any errors in your flannel pods that don't get the podCIDR assigned to them? — Rico, Oct 03 '18 at 19:23
Hi, yes. The kube-flannel-ds pod keeps starting, failing, and restarting. Adding the log output as an edit to my post. — user10452751, Oct 03 '18 at 23:09
provide your k8s component versions and share information on how you enabled flannel. I have recreated your case(v1.12.0). 1) kubeadm init --pod-network-cidr=10.244.0.0/16; 2)kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml; 3)joined 4 nodes - all of them are up and running without issues and all have proper podCIDR — Vit, Oct 04 '18 at 13:14
I've done pretty much the same as you described, except my Kubernetes version is v1.11.2. I passed the pod-network-cidr in kubeadm init, applied the same yaml, and the first worker node I brought up worked fine, but subsequent nodes present the issue I described. — user10452751, Oct 04 '18 at 17:03

score 6 · Answer 1 · answered Oct 05 '18 at 20:27

I was able to solve my issue. In my multi-master setup, on one of my master nodes, the kube-controller-manager.yaml (in /etc/kubernetes/manifest) file was missing the two following fields:

--allocate-node-cidrs=true
--cluster-cidr=10.244.0.0/16

Once adding these fields to the yaml, I reset the kubelet service and everything was working great when trying to add a new worker node.

This was a mistake on my part, because when initializing one of my master nodes with kubeadm init, I must of forgot to pass the --pod-network-cidr. Oops.

Hope this helps someone out there!

score 1 · Answer 2 · answered Jan 23 '21 at 00:37

If you only have a couple of worker nodes like I did rather than kubeadm revert and then passing in the kubeadm init --pod-network-cidr=10.244.0.0/16 you can do the following on each node and issue should disappear:

kubectl patch node node2-worker -p '{"spec":{"podCIDR":"10.244.0.0/16"}}'

where node2-worker is your node name and not the one shown.

score 0 · Answer 3 · edited Jan 27 '20 at 21:41

I'm using kubernetes v1.16 with docker-ce v17.05. The thing is, I only have one master node, which is inited with --pod-network-cidr option.

The flannel pod on another worker node failed to syncing, according to kubelet log under /var/log/message. Checking this pod (with docker logs <container-id>), it turned out that node <NODE_NAME> pod cidr not assigned.

I fixed it by manually set the podCidr to the worker node, according to this doc

Although I've not yet figured out why this manually set-up is required, because as the the doc pointed out:

If kubeadm is being used then pass --pod-network-cidr=10.244.0.0/16 to kubeadm init which will ensure that all nodes are automatically assigned a podCIDR.

Kubernetes worker nodes not automatically being assigned podCidr on kubeadm join

3 Answers3