My cluster was working fine before and no changes was made to it. But a few days ago I noticed that the newly scheduled nodes are all in NotReady state. Here is what I could find in the kubelet log:
4069 cni.go:240] "Unable to update cni config" err="no networks found in /etc/cni/net.d"
4069 kubelet.go:2398] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized"
An ls -la /etc/cni/net.d shows an empty directory. And I found that VPC CNI ADD-ON in in degraded state so I tried upgrade and remove it then recreate it. But no matter what I do to upgrade/recreate it, the add-on is always stuck in upgrading or creating state forever until it times out and gives me creation failed status.
AMI used: 1.23.17-20230711 k8s version: 1.23 vpc cni version: 1.10.4
I don't think it's the version incompatibility because I have other clusters running on the exact same version of k8s and vpc.
Been stuck on this for days. Any help would be appreciated. Thanks in advance!
