2

I installed kind to play around with Kubernetes.

If I use top and sort by CPU usage (key C), then I see that kube-apiserver is constantly consuming 5 to 10% CPU.

Why?

I don't have installed something up to now:

guettli@p15:~$ kubectl get pods --all-namespaces
NAMESPACE            NAME                                         READY   STATUS    RESTARTS   AGE
kube-system          coredns-558bd4d5db-ntg7c                     1/1     Running   0          40h
kube-system          coredns-558bd4d5db-sx8w9                     1/1     Running   0          40h
kube-system          etcd-kind-control-plane                      1/1     Running   0          40h
kube-system          kindnet-9zkkg                                1/1     Running   0          40h
kube-system          kube-apiserver-kind-control-plane            1/1     Running   0          40h
kube-system          kube-controller-manager-kind-control-plane   1/1     Running   0          40h
kube-system          kube-proxy-dthwl                             1/1     Running   0          40h
kube-system          kube-scheduler-kind-control-plane            1/1     Running   0          40h
local-path-storage   local-path-provisioner-547f784dff-xntql      1/1     Running   0          40h


guettli@p15:~$ kubectl get services --all-namespaces
NAMESPACE     NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP                  40h
kube-system   kube-dns     ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   40h


guettli@p15:~$ kubectl get nodes
NAME                 STATUS   ROLES                  AGE   VERSION

kind-control-plane   Ready    control-plane,master   40h   v1.21.1
guettli@p15:~$ kubectl get nodes --all-namespaces
NAME                 STATUS   ROLES                  AGE   VERSION
kind-control-plane   Ready    control-plane,master   40h   v1.21.1

I am curious. Where does the CPU usage come from? How can I investigate this?

guettli
  • 25,042
  • 81
  • 346
  • 663
  • doesn't kubelet reports to kube-apiserver various statutes about the node on timed bases? I am not versatile enough to know what actually happens, but it seems normal that it consumes cpu for period things it does. There might be more things I am not aware of – Eugene Jan 05 '22 at 12:52

1 Answers1

5

Even in an empty cluster with just one master node, there are at least 5 components that reach out to the API server on a regular basis:

  • kubelet for the master node
  • Controller manager
  • Scheduler
  • CoreDNS
  • Kube proxy

This is because API Server acts as the only entry point for all components in Kubernetes to know what the cluster state should be and take action if needed.

If you are interested in the details, you could enable audit logs in the API server and get a very verbose file with all the requests being made. How to do so is not the goal of this answer, but you can start from the apiserver documentation.

whites11
  • 12,008
  • 3
  • 36
  • 53
  • the suggestion about audit logs is great. sort of dumb that I [have implemented some functionality for this exact purpose in spring](https://github.com/spring-cloud/spring-cloud-kubernetes/pull/898), but totally for got to mention in the comment :) 1+ – Eugene Jan 05 '22 at 13:24