0

I have deployed 3 node external ETCD database (etcdctl version: 3.4.7) cluster for my Kubernetes v1.18.6 cluster using etcdadm tool. my certificate is expring in couple of months.

I believe kubeadm alpha certs renew all command will renew kubernetes certificates. May Know the correct steps to renew external ETCD database cluster certificate?

my cluster certificate details

# kubeadm alpha certs check-expiration 

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Jul 20, 2021 14:13 UTC   152d                                    no
apiserver                  Jul 20, 2021 14:13 UTC   152d            ca                      no
apiserver-kubelet-client   Jul 20, 2021 14:13 UTC   152d            ca                      no
controller-manager.conf    Jul 20, 2021 14:13 UTC   152d                                    no
front-proxy-client         Jul 20, 2021 14:13 UTC   152d            front-proxy-ca          no
scheduler.conf             Jul 20, 2021 14:13 UTC   152d                                    no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Apr 17, 2030 01:19 UTC   9y              no
front-proxy-ca          Apr 17, 2030 01:19 UTC   9y              no

Master node certificate details

/etc/kubernetes/pki/ca.crt,             Apr 17 01:19:52 2030 GMT
/etc/kubernetes/pki/apiserver.crt,             Jul 20 14:13:09 2021 GMT
/etc/kubernetes/pki/apiserver-kubelet-client.crt,             Jul 20 14:13:10 2021 GMT
/etc/kubernetes/pki/front-proxy-ca.crt,             Apr 17 01:19:52 2030 GMT
/etc/kubernetes/pki/front-proxy-client.crt,             Jul 20 14:13:10 2021 GMT


/etc/etcd/pki/ca.crt,             Apr 17 01:19:35 2030 GMT
/etc/etcd/pki/server.crt,             Apr 19 01:19:36 2021 GMT
/etc/etcd/pki/peer.crt,             Apr 19 01:19:36 2021 GMT
/etc/etcd/pki/etcdctl-etcd-client.crt,             Apr 19 01:19:36 2021 GMT
/etc/etcd/pki/apiserver-etcd-client.crt,             Apr 19 01:19:36 2021 GMT

Thanks SR

sfgroups
  • 243
  • 2
  • 4
  • 14
  • The devil's in the details about that kind of stuff; we just nuke the etcd member and let the ASG recreate a new one, solving the rotation problem on the regular. If your apiservers are more pets, I can see how that would be a bigger PITA. _Back on topic:_ what have you already tried, and what error is it producing for you? – mdaniel Feb 19 '21 at 06:01
  • @mdaniel since this cluster is used, I am collecting the steps before trying to renew the certificates. – sfgroups Feb 19 '21 at 23:44

2 Answers2

1

I used these steps to renew the certs on kubernets cluster v1.18.6.

  1. compile the etcdadm cert branch code

    git clone -b cert https://github.com/pytimer/etcdadm.git
     cd  etcdadm
     docker run --rm -it -v "$PWD":/etcdadm  golang bash
     cd /etcdadm
     make
     cp etcdadm etcdadm-cert 
    
  2. copy etcdadm-cert file to all three server.

  3. Renew the cert on first master

    /opt/bin/etcdadm-cert certs renew kubeadm alpha certs renew all

  4. Reboot the first master

  5. check the etcd member and kubernetes certificate expire data

Repeated step 2 to 5 on ther master nodes

use these commands to validate

/opt/bin/etcdctl.sh   member list
kubeadm alpha certs check-expiration

Thanks SR

sfgroups
  • 243
  • 2
  • 4
  • 14
0

For anyone else who's come across this topic (I assume the OP has already solved this):

If you have deployed your certificates with kubeadm and they are in the right place, you can do exactly the same thing as you do with the other kubernetes nodes:

kubeadm alpha certs renew all

And this will renew all certificates that kubeadm identifies and can renew (can renew = it as a private key for).

If you've configured the external etcd cluster like in the kubernetes documentation, then the private key associated with etcd (/etc/kubernetes/pki/etcd/ca.key) will most probably only on the first etcd node. So that means that you can only renew the etcd-related certificates there.

The shortest way to fix this is simply to copy this key to all other nodes to the correct, default path (mentioned above) and kubeadm should be able to renew all certificates and you're good to go. You don't need to reset the etcd containers, they will automatically reread the certificates.

For newer versions of kubeadm kubeadm certs renew all (without "alpha") should work.

You can do the same for the rest of the kubernetes nodes, which, of course, need access to the etcd API. Normally though you're supposed to restart the controlplane nodes pods in kube-system (apiserver, controller-manager, scheduler).

Lethargos
  • 455
  • 2
  • 7
  • 19