1

I have been able to generate the following output with plugin of kubectl in following way :

./kubectl get namespaces | awk '{ print $1 }' | while read x; do ./kubectl resource-capacity --sort cpu.util --namespace $x --util --pod-count --pods; done

Error getting Pod Metrics: pods.metrics.k8s.io is forbidden: User "u-567" cannot list resource "pods" in API group "metrics.k8s.io" in the namespace "NAME"
For this to work, metrics-server needs to be running in your cluster

NODE                                        POD                                                               CPU REQUESTS   CPU LIMITS      CPU UTIL    MEMORY REQUESTS   MEMORY LIMITS   MEMORY UTIL   POD COUNT
*                                           *                                                                 16550m (16%)   45060m (45%)    0Mi (0%)    38174Mi (9%)      33492Mi (8%)    0Mi (0%)      16/1540

ip-10-11-7-77.eu-west-1.compute.internal    *                                                                 6150m (76%)    14160m (177%)   139m (1%)   16784Mi (53%)     9632Mi (30%)    3126Mi (9%)   5/110
ip-10-11-7-77.eu-west-1.compute.internal    anomaly-detector-water-transformer-default-msxhv-deploymenrnv85   1035m (12%)    3040m (38%)     46m (0%)    2138Mi (6%)       2376Mi (7%)     300Mi (0%)
ip-10-11-7-77.eu-west-1.compute.internal    anomaly-detector-blades-transformer-default-gmvb2-deploymezzt5r   1035m (12%)    3040m (38%)     45m (0%)    2138Mi (6%)       2376Mi (7%)     292Mi (0%)
ip-10-11-7-77.eu-west-1.compute.internal    leadec-uk-inference-m1-predictor-default-zvxkb-deployment-8mscd   1035m (12%)    3040m (38%)     43m (0%)    2138Mi (6%)       2376Mi (7%)     277Mi (0%)
ip-10-11-7-77.eu-west-1.compute.internal    datascience-0                                                     2010m (25%)    2000m (25%)     8m (0%)     8232Mi (26%)      128Mi (0%)      2259Mi (7%)
ip-10-11-7-77.eu-west-1.compute.internal    anomaly-detector-leadec-predictor-default-tqjsg-deployment4bqdc   1035m (12%)    3040m (38%)     0Mi (0%)    2138Mi (6%)       2376Mi (7%)     0Mi (0%)

ip-10-11-7-134.eu-west-1.compute.internal   *                                                                 2070m (25%)    6080m (76%)     88m (1%)    4276Mi (13%)      4752Mi (15%)    597Mi (1%)    2/110
ip-10-11-7-134.eu-west-1.compute.internal   leadec-uk-inference-m1-transformer-default-84tk6-deploymeng4qjf   1035m (12%)    3040m (38%)     45m (0%)    2138Mi (6%)       2376Mi (7%)     298Mi (0%)
ip-10-11-7-134.eu-west-1.compute.internal   leadec-uk-synthetic-f65535db1d5e1078fca102b9506391d8-deplop74ww   1035m (12%)    3040m (38%)     44m (0%)    2138Mi (6%)       2376Mi (7%)     299Mi (0%)

ip-10-11-7-123.eu-west-1.compute.internal   *                                                                 1035m (12%)    3040m (38%)     55m (0%)    2138Mi (6%)       2376Mi (7%)     317Mi (1%)    1/110
ip-10-11-7-123.eu-west-1.compute.internal   leadec-uk-synthetic-58a5440150937e2198bfdd610a109b95-deplomfn97   1035m (12%)    3040m (38%)     55m (0%)    2138Mi (6%)       2376Mi (7%)     317Mi (1%)

ip-10-11-7-190.eu-west-1.compute.internal   *                                                                 1085m (13%)    3540m (44%)     50m (0%)    2148Mi (6%)       2476Mi (7%)     327Mi (1%)    2/110
ip-10-11-7-190.eu-west-1.compute.internal   leadec-uk-synthetic-year3-m4-predictor-default-ttdkp-deplovvtj6   1035m (12%)    3040m (38%)     47m (0%)    2138Mi (6%)       2376Mi (7%)     267Mi (0%)
ip-10-11-7-190.eu-west-1.compute.internal   random-experiment-random-568ff69c58-j7lx4                         50m (0%)       500m (6%)       3m (0%)     10Mi (0%)         100Mi (0%)      60Mi (0%)

ip-10-11-7-44.eu-west-1.compute.internal    *                                                                 1035m (12%)    3040m (38%)     46m (0%)    2138Mi (6%)       2376Mi (7%)     268Mi (0%)    1/110
ip-10-11-7-44.eu-west-1.compute.internal    leadec-uk-synthetic-year4-m4-predictor-default-7d9cb-deplofc5kf   1035m (12%)    3040m (38%)     46m (0%)    2138Mi (6%)       2376Mi (7%)     268Mi (0%)

ip-10-11-7-57.eu-west-1.compute.internal    *                                                                 1035m (12%)    3040m (38%)     46m (0%)    2138Mi (6%)       2376Mi (7%)     290Mi (0%)    1/110
ip-10-11-7-57.eu-west-1.compute.internal    anomaly-detector-leadec-transformer-default-7j42s-deploymeqhxtd   1035m (12%)    3040m (38%)     46m (0%)    2138Mi (6%)       2376Mi (7%)     290Mi (0%)

ip-10-11-7-113.eu-west-1.compute.internal   *                                                                 1035m (25%)    3040m (76%)     43m (1%)    2138Mi (13%)      2376Mi (15%)    298Mi (1%)    1/110
ip-10-11-7-113.eu-west-1.compute.internal   leadec-uk-synthetic-m4-transformer-default-9d4pz-deploymenhtfsv   1035m (25%)    3040m (76%)     43m (1%)    2138Mi (13%)      2376Mi (15%)    298Mi (1%)

ip-10-11-7-168.eu-west-1.compute.internal   *                                                                 0Mi (0%)       0Mi (0%)        0Mi (0%)    0Mi (0%)          0Mi (0%)        0Mi (0%)      0/110

I want to append namespace in each loop as column, how can i do the same.

Thanks in advance

utkarsh saraf
  • 49
  • 7
  • 1
    `kubectl describe node | perl -0777 -wnE '@pods = /Name:\s+([^ ]+)\n.*?Non-terminated\s+Pods:\s+\([0-9]+\s+in\s+total\)\n(.*?)\nAllocated resources:/gs;say for @pods'` should to the work without any plugin, although the output would look bit different but it would have everything you have in above output. – P.... Sep 07 '22 at 22:09
  • @P.... thanks a ton...i am not much aware of perl so can you do just one more favour..write each section of each node to seperate csv..it will really help – utkarsh saraf Sep 08 '22 at 08:12

1 Answers1

2

The answer is adjusted as per the comments by OP.

The command to print the pod resources by node:

kubectl describe  node | perl -0777 -wnE '@pods = /Name:\s+([^ ]+)\n.*?Non-terminated\s+Pods:\s+\([0-9]+\s+in\s+total\)\n(.*?)\nAllocated resources:/gs;say for @pods'

1st step, Write the output of each node to their respective file.

kubectl describe  node | perl -0777 -wnE '@pods = /Name:\s+([^ ]+)\n.*?Non-terminated\s+Pods:\s+\([0-9]+\s+in\s+total\)\n(.*?)\nAllocated resources:/gs;say for @pods'|awk -v OFS=',' '/^[^ ]+/{node=$0;next} {print  $0 > node ".csv"}'

The above command would create the files like below:

ls -lrt *.csv 
-rw-rw-r-- 1 p.. p.. 2800 Sep  8 12:43 dev-kube-worker-3.csv
-rw-rw-r-- 1 p.. p.. 2782 Sep  8 12:43 dev-kube-worker-2.csv
-rw-rw-r-- 1 p.. p.. 2800 Sep  8 12:43 dev-kube-worker-1.csv
-rw-rw-r-- 1 p.. p.. 1551 Sep  8 12:43 dev-kube-controller-1.csv

Final modification: However, the content of these files would not be in CSV format, so to make the content to be CSV:

kubectl describe  node | perl -0777 -wnE '@pods = /Name:\s+([^ ]+)\n.*?Non-terminated\s+Pods:\s+\([0-9]+\s+in\s+total\)\n(.*?)\nAllocated resources:/gs;say for @pods'|awk -v OFS=',' '/^[^ ]+/{node=$0;next} {$1=$1;print  $0 > node ".csv"}'
P....
  • 17,421
  • 2
  • 32
  • 52