How to get the exit code from playwright?

Question

I'm running some playwright tests in a pipeline but I'm having problems trying to fail the build when a test fails.

In my pipeline I call a make file that calls a shell script that installs playwright, does some setup, and then runs the command yarn playwright test the script I call always returns an exit code 0 but Playwright returns an exit code 1 if a test fails.

Is it possible for me to get the exit code of Playwright so I can then pass it back up and handle it in my pipeline?

Update: The line in my makefile that calls the sh script is:

uiTests: node-install node-audit
    $(DOCKER_RUN_NODE) bash /app/scripts/playwright.sh

My Playwright.sh script:

  #!/usr/bin/env bash

apt-get update
apt-get install -y gnupg2 gnupg1 wget
docker pull mcr.microsoft.com/playwright/python:v1.27.0-focal
yarn install
yarn playwright install 
yarn playwright install-deps
yarn run local & yarn run uiTests
rm -rf node_modules

This pulls the dependencies then starts the local instance of the site and runs the tests

yarn run uiTests runs this command "BASE_URL='http://localhost:3000' yarn playwright test"

You're going to have to provide a lot more detail than this if you want an answer. If your shell script isn't exiting with the correct error code that means that you didn't preserve the error code that `yarn` generated. — MadScientist, Jan 18 '23 at 18:50
First: Put `|| exit` after each command that should abort the script if it fails (this is the more reliable/consistent alternative to the [deeply unreliable](https://mywiki.wooledge.org/BashFAQ/105#Exercises) shortcut `set -e`, and will make the script's exit status be the same as the individual command that failed) — Charles Duffy, Jan 18 '23 at 21:37
Second: After each backgrounded command, collect `$!` to get its PID; if you, say, do `yarn run local & yarn_local_pid=$!`, then you can later `wait "$yarn_local_pid" || exit` and that'll make your script exit with the same status as the failed command. — Charles Duffy, Jan 18 '23 at 21:38
Finally: If you want `rm -rf node_modules` to happen when your script exits even if the script is exiting early on account of an error, think about moving it from being the last command to instead be a trap. `trap 'rm -rf node_modules' EXIT`, f/e. — Charles Duffy, Jan 18 '23 at 21:40
BTW, think about dropping the `sh` extension and the explicit `bash` invocation both. If your file permissions are right (you have `+x`), you can just have your makefile invoke `/app/scripts/playwright` and let the shebang choose the appropriate interpreter; that way if you rewrite your script to need a different interpreter in the future, that one line is the only thing that needs to change. See also [commandname extensions considered harmful](https://www.talisman.org/~erlkonig/documents/commandname-extensions-considered-harmful/) — Charles Duffy, Jan 18 '23 at 21:41
Charles' suggestions are spot-on. But just to be clear, unless you take some steps to do something different such as Charles suggests, the exit code of a shell script will be the exit code of the last command in the shell script. So, in your case if the `rm ...` command exits with success (0) then the script exits with success. If the `rm ...` command exits with failure (non-0) then the script exits with failure. It's not the case that all the commands will be run and the "worst" exit code will be remembered and used as the exit code of the script, or something like that. — MadScientist, Jan 19 '23 at 05:15

Cees Timmerman · Answer 1 · 2023-07-23T21:24:52.433

A & B runs command A in the background. You appear to want A && B, which runs command B if command A succeeded (0 exit code). If you want to run B if A failed, use A || B. If you always want to run B, use A; B or newline instead of ;.

For example:

❯ output=$(npx playwright test); code=$?; echo "Playwright output is $output, with exit code $code"
Playwright output is 
Running 2 tests using 2 workers
  1) [chromium] › chat.spec.ts:3:5 › Chat only for logged in users ─────────────────────────────────

    Test timeout of 10000ms exceeded.
[...]
  1 failed
    [chromium] › chat.spec.ts:3:5 › Chat only for logged in users ──────────────────────────────────
  1 passed (10.9s), with exit code 1

If you also want error output, use 2>&1:

❯ output=$(datee 2>&1); code=$?; if ((code != 0)); then echo "Output is $output, with exit code $code"; fi
Output is zsh: command not found: datee, with exit code 127

How to get the exit code from playwright?

1 Answers1