0

I created a k8s installed by k0s on the aws ec2 instance. In order to make delivery new cluster faster, I try to make an AMI for it.

However, I started a new ec2 instance, the internal IP changed and the node become NotReady

ubuntu@ip-172-31-26-46:~$ k get node
NAME               STATUS     ROLES    AGE   VERSION
ip-172-31-18-145   NotReady   <none>   95m   v1.21.1-k0s1
ubuntu@ip-172-31-26-46:~$

Would it be possible to reconfigure it ?

Work around

I found a work around to make the AWS AMI working

Short answer

  1. install node with kubelet's --extra-args
  2. update the kube-api to the new IP and restart the kubelet

Details :: 1

In the kubernete cluster, the kubelet plays the node agent node. It will tell kube-api "Hey, I am here and my name is XXX".

The name of a node is its hostname and could not be changed after created. It could be set by --hostname-override.

If you don't change the node name, the kube-api will try to use the hostname then got errors caused by old-node-name not found.

Details :: 2

To k0s, it put kubelet' KUBECONFIG in the /var/lib/k0s/kubelet.conf, there was a kubelet api server location

server: https://172.31.18.9:6443

In order to connect a new kube-api location, please update it

qrtt1
  • 7,746
  • 8
  • 42
  • 62

1 Answers1

1

Did you check the kubelet logs? Most likely it's a problem with certificates. You cannot just make an existing node into ami and hope it will work since certificates are signed for specific IP.

Check out the awslabs/amazon-eks-ami repo on github. You can check out how aws does its k8s ami.

There is a files/bootstrap.sh file in repo that is run to bootstrap an instance. It does all sort of things that are instance specific which includes getting certificates.

If you want to "make delivery new cluster faster", I'd recommend to create an ami with all dependencies but without an actual k8s boostraping. Install the k8s (or k0s in your case) after you start the instance from ami, not before. (Or figure out how to regenerate certs and configs that are node specific.)

Matt
  • 7,419
  • 1
  • 11
  • 22