How to use `set -e` inside a bash command substitution?

Question

I have a simple shell script with the following preamble:

#!/usr/bin/env bash
set -eu
set -o pipefail

I also have the following function:

foo() {
  printf "Foo working... "
  echo "Failed!"
  false  # point of interest #1
  true   # point of interest #2
}

Executing foo() as a regular command works as expected: The script exits at #1, because the return code of false is non-zero and we use set -e.

My goal is to capture the output of the function foo() in a variable, and only print it in case an error occurs during the execution of foo(). This is what I've come up with:

printf "Doing something that could fail... "
if a="$(foo 2>&1)"; then
  echo "Success!"
else
  code=$?
  echo "Error:"
  printf "${a}"
  exit $code
fi

The script doesn't exit at #1 and the "Success!" path of the if statement is executed. Commenting out the true at #2 causes the "Error:" path of the if statement to be executed.

It seems like bash just ignores set -e inside the substitution and the if statement is simply checking the return code of the last command in foo().

Q: ~~What causes this weird behaviour?~~

A: This is just how bash works, it's normal behaviour

Q: ~~Is there any way to make bash respect set -e inside a command substitution and make this work correctly?~~

A: You shouldn't use set -e for this purpose

Q: ~~How would you go about implementing this without set -e (i.e. print the output of a function only if something went wrong while executing it)?~~

A: See accepted answer and my "final thoughts" section.

I am using:

GNU bash, version 5.0.11(1)-release (x86_64-apple-darwin18.6.0)

Final thoughts / takeaway (might be useful for someone else):

Beware that using if ...; then, or even && ... || ... will disable most kinds of "traditional" bash error handling methods (this includes set -e and trap ... ERR + set -o errtrace) by design. If you want to do something like I did, you probably should check the return codes inside your function manually and return a non-null exit code by hand (dangerous_command || return 1) to avoid continuing execution on errors (you can do this whether you use set -e or not).

As answered, set -e does not propagate inside command substitutions by design. If you wish to implement error handling logic which does, you can use trap ... ERR in combination with set -o errtrace, which will work with functions running inside command substitutions (that is unless you put them inside an if statement, which will disable trap ... ERR as well, so in this case manual return code checking is your only option if you wish to stop your function on errors).

If you think about it, this whole behaviour kind of makes sense: you wouldn't expect your script to terminate on a command "guarded" by an if statement, as the whole point of your if statement is checking whether the command succeeds or not.

Personally I still wouldn't go as far as avoiding set -e and trap ... ERR entirely as they can be really useful, but understanding how they behave in different circumstances is important, because they are no silver bullet either.

It's documented to work like that. Read about `-e` under https://www.gnu.org/software/bash/manual/bash.html#The-Set-Builtin — glenn jackman, Nov 14 '19 at 15:12
IMO, you should not rely on set -e, but use conditional commands for conditional execution. — glenn jackman, Nov 14 '19 at 15:12
In this case, bash is *not* ignoring `set -e`. It is behaving as designed. Unfortunately, the way it is designed prevents you from using it the way you want. It's a kludge on a bandaid. Just don't use it. — William Pursell, Nov 14 '19 at 15:52
`false` is not `return false`. You are returning zero, thus the parent shell is thinking your function succeeded. — KamilCuk, Nov 14 '19 at 16:10

score 7 · Accepted Answer · edited Jun 20 '20 at 09:12

Q: How would you go about implementing this without set -e (i.e. print the output of a function only if something went wrong while executing it)?

You may use this way by checking return value of the function:

#!/usr/bin/env bash

foo() {
  local n=$RANDOM
  echo "Foo working with random=$n ..."
  (($n % 2))
}

echo "Doing something that could fail..."
a="$(foo 2>&1)"
code=$?
if (($code == 0)); then
  echo "Success!"
else
  printf '{"ErrorCode": %d, "ErrorMessage": "%s"}\n' $code "$a"
  exit $code
fi

Now run it as:

$> ./errScript.sh
Doing something that could fail...
Success!
$> ./errScript.sh
Doing something that could fail...
{"ErrorCode": 1, "ErrorMessage": "Foo working with random=27662 ..."}
$> ./errScript.sh
Doing something that could fail...
Success!
$> ./errScript.sh
Doing something that could fail...
{"ErrorCode": 1, "ErrorMessage": "Foo working with random=31864 ..."}

This dummy function code returns failure if $RANDOM is even number and success for $RANDOM being odd number.

Original answer for original question

You need to enable set -e in command substitution as well:

#!/usr/bin/env bash
set -eu
set -o pipefail

foo() {
  printf "Foo working... "
  echo "Failed!"
  false  # point of interest #1
  true   # point of interest #2
}

printf "Doing something that could fail... "
a="$(set -e; foo)"
code=$?
if (($code == 0)); then
  echo "Success!"
else
  echo "Error:"
  printf "${a}"
  exit $code
fi

Then use it as:

./errScript.sh; echo $?

Doing something that could fail... 1

However do note that using set -e is not ideal in shell scripts and it may fail to exit script in many scenarios.

Do check this important post on set -e

The code under **Original answer for original question** doesn't work (tested with Bash 4.2). Since `errexit` is set in the preamble, the error status from `foo` causes the program to exit on the `a="$(set -e; foo)"` line. When `errexit` is active there is normally no point in storing or checking the value of `$?` because the program has already exited if it was non-zero. — pjh, Nov 14 '19 at 20:27
Yes that's right. It will work only when `set -e` is not set in calling script. — anubhava, Nov 14 '19 at 20:40

score 1 · Answer 2 · answered Nov 14 '19 at 16:11

How would you go about implementing this without set -e (i.e. print the output of a function only if something went wrong while executing it)?

Return a nonzero return status from your function to indicate an error/failure.

foo() {
  printf "Foo working... "
  echo "Failed!"
  return 1  # point of interest #1
  return 0   # point of interest #2
}

if a="$(foo 2>&1)"; then
  echo "Success!"
else
  code=$?
  echo "Error:"
  printf "${a}"
  exit $code
fi

pjh · Answer 3 · 2019-11-14T21:40:01.563

As others have stated, errexit is not a reliable way to deal with errors in programs. Just one of the big problems with it is that it is silently disabled in several common situations, including within command substitution.

If you still want to use errexit, there are a few ways to get the effect that you want.

One way to do it is to temporarily disable errexit in the main code, explicitly enable errexit within the command substitution (as demonstrated in the answer by @anubhava), get the exit code of the command substitution from $?, and re-enable errexit in the main code.

Another possible way to do it (after the preamble and foo definition code in the question) is:

shopt -s lastpipe

printf "Doing something that could fail... "
set +o pipefail
foo 2>&1 | { read -r -d '' a || true; }
code=${PIPESTATUS[0]}
set -o pipefail

if (( code == 0 )); then
    echo "Success!"
else
    echo "Error:"
    printf '%s\n' "$a"
    exit "$code"
fi

shopt -s lastpipe causes the last command of pipelines to be run in the top-level shell. It means that variables set in commands at the end of pipelines (like a in this case) can be used later in the program. lastpipe was introduced in Bash 4.2 so this code won't work with older versions of Bash.
The set +o pipefail (temporarily) disables pipefail to prevent a failing foo at the start of a pipeline causing the whole pipeline to fail.
The read -r -d '' a reads all of its input (assumed not to contain a NUL character), including internal newlines, into the variable a.
The { ... || true; } around the read hides the non-zero status returned by read when it encounters EOF on its input, thus preventing the pipeline from failing.
code=${PIPESTATUS[0]} captures the status of the first command in the pipeline (foo).
set -o pipefail re-enables pipefail so it is enabled for the rest of the program.
A few tweaks have been made to the code in the question to stop Shellcheck warnings.

How to use `set -e` inside a bash command substitution?

3 Answers3

Original answer for original question