Running `kubectl exec` script from file is not working

Question

I'm trying to create a script in order to run multiple kubectl exec commands against multiple pods with multiple containers. The script seems to generate the command just fine but errors out when attempting to run it.

example command that is generated: kubectl -n <namespace> exec <pod_name> -c <container_name> -- openssl version

When I copy the generated command and run it directly it works fine, but if I try to run the command within the script I get an error.

OCI runtime exec failed: exec failed: unable to start container process: exec: "openssl version": executable file not found in $PATH: unknown

command terminated with exit code 126

snippet from .sh file:

for pod in $PODS; do
CONTAINERS=($(kubectl -n $NAMESPACE get pods $pod -o jsonpath='{.spec.containers[*].name}' | tr -s '[[:space:]]' '\n'))
header "{pod: \"$pod\", containers: \"$(echo $CONTAINERS | tr -d '\n')\"}"

if [ "$DRYRUN" != "true" ]; then
    for container in $CONTAINERS; do
        echo "COMMAND BEING RUN: \"kubectl -n $NAMESPACE exec $pod -c $container -- $COMMAND\""
        kubectl -n $NAMESPACE exec $pod -c $container -- $COMMAND
    done
fi
done

Maybe pathing is incorrect when you are trying directly to run openssl. Have you tried to add a full path to your command? — Dawid Kruk, Oct 31 '22 at 19:02
based only upon the pasted output, I'm guessing you have quoted `COMMAND` so oci is looking for a binary named `"openssl command"`; running your script with `set -x` will show what it's actually substituting for `$COMMAND`, or you can cheat and `... -c $container -- sh -c $COMMAND` if that starts to work, then my theory was correct — mdaniel, Oct 31 '22 at 19:10
tried with `-- sh -c $COMMAND` and getting the same error. Tried setting the full path as well but didn't work. — cybersnow1989, Oct 31 '22 at 20:03
@cybersnow1989 see my answer posted below for the working fix, I ran into the exact same issue as you trying to run `openssl version` against all containers in all pods in all deployments — Othyn, Nov 02 '22 at 10:43

score 3 · Answer 1 · answered Oct 31 '22 at 20:09

3

The command provided with exec will be executed without a shell. This means that environment variables will not be available. Thus the error specifying missing executable path in $PATH. Only executables installed in default $PATH (like /bin/) can be executed.

You can load the shell and provide $command as a command to be executed;

kubectl exec -n $NAMESPACE --container $container -it $pod -- /bin/sh -c "$COMMAND"

answered Oct 31 '22 at 20:09

Shivam Nagar

154
5

ran the above and am getting `kubectl exec -n account-engagement --container aec-consumer -it account-engagement-aec-consumer-66c774946f-qkhhx -- /bin/sh -c openssl version /bin/sh: openssl: not found command terminated with exit code 127` – cybersnow1989 Nov 01 '22 at 14:49
when running with `set -x` however it seems that my command is being run with single quotes surrounding, that may be an issue. – cybersnow1989 Nov 01 '22 at 14:51

Othyn · Accepted Answer · 2022-11-02T10:47:51.370

Haha, this is exactly the problem I'm trying to solve. By any chance, are you trying to check OpenSSL versions on all containers due to the latest CVE's dropping yesterday? Because thats exactly what I'm trying to do.

Anyway, here is the solution - a problem in which I've had before and its down to the way sh/bash/zsh interprets strings.

It's hard to debug, as I'd need to see you're whole script instead of just the pod exec loop, but the underlying problem is: $COMMAND cannot be placed into a string or used as a string, otherwise your shell script will wrap it into a string and kubectl will interpret that as the whole thing being a command instead of correctly forwarding the breakpoints.

This is evident by the error message, its trying to interpret openssl version literally as a whole string that makes up the binary name you are trying to call, in which obviously openssl version isn't a command that will be on your $PATH, instead of correctly interpreting each command in the string and executing openssl which is on your $PATH with the version argument against it.

As I'm not sure if I've explained that very well, so to assist, here are some visual/working examples.

This will not work:

# ./my-script.sh openssl version
# Will OCI error as "openssl version" cannot be found in the containers $PATH

kubectl exec "${pod}" -c "${container}" -- "${@:1}"

COMMAND="${@:1}"

kubectl exec "${pod}" -c "${container}" -- $COMMAND

COMMAND=${@:1}

kubectl exec "${pod}" -c "${container}" -- "${COMMAND}"

This will work:

# ./my-script.sh openssl version
# LibreSSL 3.3.6

kubectl exec "${pod}" -c "${container}" -- ${@:1}

COMMAND=${@:1}

kubectl exec "${pod}" -c "${container}" -- $COMMAND

Simply put, if the command that you wish to forward is used as a string, it will not work.

While not exactly what I ended up with this pointed me in the right direction. Spellcheck plugin didn't like it though — cybersnow1989, Nov 02 '22 at 15:53

Running `kubectl exec` script from file is not working

2 Answers2