0

I have 10 Kubernetes nodes (consider them as VMs) which have between 7 and 14 allocatable CPU cores which can be requested by Kubernetes pods. Therefore I'd like to show cluster CPU usage.

This is my current query

sum(kube_pod_container_resource_requests_cpu_cores{node=~"$node"}) / sum(kube_node_status_allocatable_cpu_cores{node=~"$node"})

This query shows strange results, for example over 400%.

I would like to add filter to only calculate this for nodes that have Running pods, since there might be some old node definitions which are not user. I have inherited this setup, so it is not that easy for me to wrap my head around it.

Any suggestions with a query that I can try?

Bob
  • 8,392
  • 12
  • 55
  • 96

1 Answers1

1

Your current query is summing up CPU utilization of each nodes so it might show invalid data.

You can check CPU utilization of all pods in the cluster by running:

sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name!=""}[5m]))

If you want to check CPU usage of each running pod you can use using:

sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name!=""}[5m])) by (pod_name)
kool
  • 3,214
  • 1
  • 10
  • 26
  • Thanks! What if I would like to exclude the pods that are not in Running state? It looks like there are a lot of Completed pods, which are destroying this calculation – Bob Apr 22 '20 at 17:25
  • Try to use following query: `kube_pod_container_resource_requests_cpu_cores and on(pod, namespace) (kube_pod_status_phase{phase="Running"}==1)` and let me know if that's what you wanted to achieve. – kool Apr 24 '20 at 13:29
  • I am getting this error if I set that `Error: Multiple Series Error`. Do you know what that might be? – Bob Apr 24 '20 at 21:26