0

I need to split a comma-separated string into array and run k6 for each of the array values parallely. Since shell script doesn't support arrays, I m using bash command in the script. I am not able to run it as bash using Dockerfile in TeamCity.

Dockerfile:

FROM loadimpact/k6:0.34.1
COPY ./src/lib /lib
COPY ./src/scenarios /scenarios
COPY ./src/k6-run-all.sh /k6-run-all.sh
WORKDIR /
ENTRYPOINT []
RUN bash -c "./k6-run-all.sh"

Shell script:

#!/bin/bash
K6_RUN_OPTIONS=${K6_RUN_OPTIONS}
ENV_NAME=${ENV_NAME:-qa}
IS_TEST_RUN=${IS_TEST_RUN:-true}
SCENARIO_NAME=${SCENARIO_NAME:-"full-card-visa"}
GWC_PC_ID=${GWC_PC_ID}

IFS=',' read -r -a PCIds <<< "$GWC_PC_ID"
echo "Number of PC ids provided in environment variables=" ${#PCIds[@]}


if [[ ${#PCIds[@]} > 0 ]]; then
   for pcId in "$@"
   do
      ENV_NAME=$ENV_NAME RUN_OPTIONS=$SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT GWC_PC_ID=$pcId k6 run $K6_RUN_OPTIONS ''$SCENARIO/index.js'' &
   done
fi

existCode=$?
if [ $existCode -ne 0 ]; then
   echo "Scenario $SCENARIO_NAME completed with the error"
   exit $existCode
fi

Error:

 #9 [5/6] RUN bash -c "./k6-run-all.sh"
17:02:02   #9 0.356 /bin/sh: bash: not found
17:02:02   #9 ERROR: executor failed running [/bin/sh -c bash -c "./k6-run-all.sh"]: exit code: 127
17:02:02   ------
17:02:02    > [5/6] RUN bash -c "./k6-run-all.sh":
17:02:02   #9 0.356 /bin/sh: bash: not found
17:02:02   ------
17:02:02   failed to solve: executor failed running [/bin/sh -c bash -c "./k6-run-all.sh"]: exit code: 127

How to modify Dockerfile or shell script to run this shell script as bash? Previously to run it as bash script the last line of Dockerfile used to be:

CMD ["sh", "-c", "./k6-run-all.sh"]

******* Edit: **********

Updated full script after knittl's answer (current issue is after adding & for parallel runs it is not working, it is not running anything inside the for loop and it is not giving any extra error or information in logs, it is like it is skipping it):

K6_RUN_OPTIONS=${K6_RUN_OPTIONS}
ENV_NAME=${ENV_NAME:-qa}
IS_TEST_RUN=${IS_TEST_RUN:-true}
SCENARIO_NAME=${SCENARIO_NAME:-"full-card-visa"}
GWC_PC_ID=${GWC_PC_ID}

OPTIONS_VARIANT=""
if $IS_TEST_RUN; then
   OPTIONS_VARIANT="-test"
fi

SCENARIO_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
SCENARIO_PATH="${SCENARIO_DIR}/scenarios"
SCENARIO="${SCENARIO_PATH}/${SCENARIO_NAME}"
echo "Executing scenario path $SCENARIO"

SCENARIO_NAME=${SCENARIO:${#SCENARIO_PATH}+1:${#SCENARIO}}
echo "Scenario Name: $SCENARIO_NAME"
echo "Run option: $SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT"

echo "pc ids provided in environment variable=" $GWC_PC_ID

if [ -z "$GWC_PC_ID" ]
then
   ENV_NAME=$ENV_NAME RUN_OPTIONS=$SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT k6 run $K6_RUN_OPTIONS ''$SCENARIO/index.js''
else
   for pcId in $(printf '%s' "$GWC_PC_ID" | tr , ' ');
   do
      ENV_NAME=$ENV_NAME RUN_OPTIONS=$SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT GWC_PC_ID=$pcId k6 run $K6_RUN_OPTIONS ''$SCENARIO/index.js'' &
   done
fi

existCode=$?
if [ $existCode -ne 0 ]; then
   echo "Scenario $SCENARIO_NAME completed with the error"
   exit $existCode
fi
Sonali Agrawal
  • 303
  • 4
  • 14
  • 1
    NOTE: k6 0.34 is quite old already, the current version is 0.40. The correct name should be `grafana/k6` (`loadimpact/k6` is outdated and deprecated) – knittl Sep 20 '22 at 18:45
  • 1
    `sh` is not `bash`; they're two different shells. Even on a distro where `sh` is a symlink to `bash`, it runs in a compatibility mode with some features turned off when started under that name. And you need to have bash installed before you can use it. – Charles Duffy Sep 20 '22 at 20:48
  • 2
    Also, note that `[[ $foo > $bar ]]` is not a numeric comparison, and if you try to use it as one you may get some surprises. For example, `[[ 3 > 10 ]]` is true, because it does a character-by-character comparison, and `3` has a higher ordinal than `1`. The sh-compatible way to do a numeric comparison is `[ "$foo" -gt "$bar" ]`; the bash way is `(( foo > bar ))`. – Charles Duffy Sep 20 '22 at 20:52
  • updated the k6 version and removed the > bash part – Sonali Agrawal Sep 21 '22 at 10:45
  • `''$SCENARIO/index.js''` fails with blanks in the value of `$SCENARIO`. `''` does basically nothing, did you mean `"`? – knittl Sep 21 '22 at 10:51
  • Note that this has now become a very different question from previously "k6 docker containers do not run bash" vs now "my script will not run commands in parallel". If you have a new question, please ask a new question (you can link this one for context). – knittl Sep 21 '22 at 10:55
  • Umm it is giving value to me. Eg: It is printing ```Executing scenario path //scenarios/card-token-payment```. Also when I give $GWC_PC_ID as empty, it is running k6 statement correctly and for non empty input, if I remove & then it is running the else part and k6 run inside for loop as well. The issue arises for non empty input and & for loop run. – Sonali Agrawal Sep 21 '22 at 10:58
  • https://stackoverflow.com/questions/73799642/run-parallel-for-loop-to-run-k6-in-posix: opened new question and accepted your answer for this question – Sonali Agrawal Sep 21 '22 at 11:01

1 Answers1

2

k6 Docker containers do not come with bash preinstalled, but with busybox. I see two options:

  1. Create your own Docker image based off grafana/k6 and manually install bash in your image.

  2. Rewrite your script to not rely on bashisms. Should be fairly easy: split your list of tests to run into one path per line and while read -r path; do …; done them. Or if support for whitespace in filenames is not required, then for path in $(printf '%s' "$GWC_PC_ID" | tr , ' '); do …; done

    Note that your current script will return with the exit code of your last k6 process, meaning that if any other test failed but the last one was successfull, that will mask the error.

PS. Time to upgrade your base Docker image too. loadimpact/k6:0.34.1 is really old (exactly 1 year). It's better to switch to grafana/k6:0.40.0, which was released a week ago.

knittl
  • 246,190
  • 53
  • 318
  • 364
  • I upgraded the Docker image and followed point 2 using the for loop method as whitespace support not required. Can you please help me by telling me how to run the content inside for loop parallely? (It is a k6 run which I updated in the question). I was using "&" but it is not working. I think that is also bash specific not shell specific. – Sonali Agrawal Sep 21 '22 at 10:12
  • I have added the script that I have updated from this answer as an edit to the question towards the end. It is splitting it correctly into different pcId without using an array but it is not running the statement inside for loop (with or without &). – Sonali Agrawal Sep 21 '22 at 10:40
  • Minor correction. The parallel run inside for loop is not working i.e. if I add & inside for loop it is not executing. Otherwise it executes fine. So I need specifically to run it in parallel. – Sonali Agrawal Sep 21 '22 at 10:51