Checking kubernetes pod CPU and memory without using third party tool

Question

I followed the first possible solution in this page: Checking kubernetes pod CPU and memory

I tried the command:

kubectl exec pod_name -- /bin/bash

But it didn't work therefore I tried the command:

kubectl exec -n [namespace] [pod_name] -- cat test.log

I know this because when I run the command:

kubectl get pods --all-namespaces | grep [pod_name]

This is what I see:

POD_NAME

But I get this error message:

OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused "exec: \"cat\": executable file not found in $PATH": unknown
command terminated with exit code 126

Could you please let me know how to resolve this?

##UPDATE I tried the k9s tool and I also cannot see CPU, MEM of finished pods, is it normal that we cannot see CPU, MEM of the finished pods ?

k9s

Answer 1

The most straight forward way to see your pod's cpu and memory usage is by installing the metrics server, and then using kubectl top pods or kubectl top pod <pod-name>.

The metrics server's impact on the cluster is minimal and it will help you monitor your cluster.

The answer in the SO post you linked seems like an hack to me and definitely not the usual way of monitoring your pod resource usage.

Answer 2

It looks like you want to check memory and CPU usage of the Pods that are Completed.

I tried the k9s tool and I also cannot see CPU, MEM of finished pods, is it normal that we cannot see CPU, MEM of the finished pods ?

I notice it's already running but with this command kubectl top pods -n %namespace% I do see the CPU, MEM of the Running pod but not the one that was already completed.

Pod marked as Completed is no longer running (terminated) and we cannot connect to it with the kubectl exec command:

$ kubectl exec -it -n cronjob hello-1618235100-xwxkc -- bash
error: cannot exec into a container in a completed pod; current phase is Succeeded 

We can see the Pod phase using kubectl get -ojson command:

$ kubectl get pod hello-1618235100-xwxkc -n cronjob
NAME                     READY   STATUS      RESTARTS   AGE
hello-1618235100-xwxkc   0/1     Completed   0          6m11s

$ kubectl get pod hello-1618235100-xwxkc -n cronjob -ojson | grep -i phase
        "phase": "Succeeded",
   

As can be found in the Pod phase documentation:

Succeeded - All containers in the Pod have terminated in success, and will not be restarted.

It is not possible to display that Pods with kubectl top command as the metric server does not store metrics history (see: the Metrics Server documentation documentation):

Only the most recent value of each metric will be remembered. If a user needs an access to historical data they should either use 3rd party monitoring solution or archive the metrics on their own

For example, I use Prometheus + Grafana and have access to historical data of my Pods: