Getting all pods for a container, storing them in text files and then using those files as args in single command

Question

The picture above shows the list of all kubernetes pods I need to save to a text file (or multiple text files).

I need a command which:

stores multiple pod logs into text files (or on single text file) - so far I have this command which stores one pod into one text file but this is not enough since I will have to spell out each pod name individually for every pod:

$ kubectl logs ipt-prodcat-db-kp-kkng2 -n ho-it-sst4-i-ie-enf > latest.txt
I then need the command to send these files into a python script where it will check for various strings - so far this works but if this could be included with the above command then that would be extremely useful:

python CheckLogs.py latest.txt latest2.txt

Is it possible to do either (1) or both (1) and (2) in a single command?

score 4 · Answer 1 · answered Oct 27 '21 at 01:52

4

The simplest solution is to create a shell script that does exactly what you are looking for:

#!/bin/sh

FILE="text1.txt"
for p in $(kubectl get pods -o jsonpath="{.items[*].metadata.name}"); do
    kubectl logs $p >> $FILE
done

With this script you will get the logs of all the pods in your namespace in a FILE. You can even add python CheckLogs.py latest.txt

answered Oct 27 '21 at 01:52

Alonso Valdivia

Hi Alonso, thank you so much for this. I have never used scripting before but have tried the above. I get errors for not having the namespace of the environment in it when I run the script - do you know where the namespace of the environment (in my case its ho-it-dev3-i-ie-enf) should go? Ive tried : for p in $(kubectl get pods ho-it-dev3-i-ie-enf -o jsonpath="{.items[*].metadata.name}"); do kubectl logs $p Error from server (Forbidden): pods "ho-it-dev3-i-ie-enf" is forbidden: User "marcus.watts@accenture.com" cannot get resource "pods" in API group "" in the namespace "default" – marcz2007 Nov 02 '21 at 14:19
To get pods from a specific namespace, you must use `for p in $ (kubectl get pods -n your-namespace -o jsonpath = "{.items [*].metadata.name}")` – Alonso Valdivia Nov 02 '21 at 17:35

score 1 · Answer 2 · answered Oct 26 '21 at 18:12

There are various tools that could help here. Some of these are commonly available, and some of these are shortcuts that I create my own scripts for.

xargs: This is used to run multiple command lines in various combinations, based on the input. For instance, if you piped text output containing three lines, you could potentially execute three commands using the content of those three lines. There are many possible variations
arg1: This is a shortcut that I wrote that simply takes stdin and produces the first argument. The simplest form of this would just be "awk '{print $1}'", but I designed mine to take optional parameters, for instance, to override the argument number, separator, and to take a filename instead. I often use "-i{}" to specify a substitution marker for the value.
skipfirstline: Another shortcut I wrote, that simply takes some multiline text input and omits the first line. It is just "sed -n '1!p'".
head/tail: These print some of the first or last lines of stdin. Interesting forms of this take negative numbers. Read the man page and experiment.
sed: Often a part of my pipelines, for making inline replacements of text.

2 Answers2