set -e and short tests

Question

When I was new to shell scripting, I used a lot of short tests instead of if statements, like false && true.

Then later I learned using set -e, and found my scripts were dying for some reason, and they would work if I replaced the short tests with full if statements. Now, the time has gone, and I still use full if statements only.

The most interesting is that if I open an interactive shell and do the following:

set -e
false && true
echo $?

it returns 1 but the shell doesn't die!

I see that I have been wasting too many lines of code. Anyone could explain to me how can I use set -e with short tests safely, eg. without the script dying?

Answer 1

The Single UNIX Specification describes the effect of set -e as:

When this option is on, if a simple command fails for any of the reasons listed in Consequences of Shell Errors or returns an exit status value >0, and is not part of the compound list following a while, until, or if keyword, and is not a part of an AND or OR list, and is not a pipeline preceded by the ! reserved word, then the shell shall immediately exit.

As you see, a failing command in an AND list will not make the shell exit.

Using set -e

Starting shell scripts with set -e is considered a best practice, since it is usually safer to abort the script if some error occurs. If a command may fail harmlessly, I usually append || true to it.

Here is a simple example:

#!/bin/sh
set -e

# [...]
# remove old backup files; do not fail if none exist
rm *~ *.bak || true

Answer 2

You need to end each command which may fail with || and a command or command list evaluating to 0. Using || will trigger your command if the expression before operator does not evaluate to 0. Your command needs to evaluate to 0 to not kill the shell.

Example:

set -e
false || true # silently ignore error

false || echo "Command failed, but exit status of echo is 0. Continuing..."

false || {                                                                      
  echo "Command failed. Continuing..."                                          
  # do something else                                                           
  false && true # all commands in the list need to be true                      
}

false && true does not cause an exit because ending expression is evaluated and evaluates to 0. From the bash man page for set -e:

Exit immediately if a pipeline (which may consist of a single simple command), a subshell command enclosed in parentheses, or one of the commands executed as part of a command list enclosed by braces (see SHELL GRAMMAR above) exits with a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test following the if or elif reserved words, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted with !. A trap on ERR, if set, is executed before the shell exits. This option applies to the shell environment and each subshell environment separately (see COMMAND EXECUTION ENVIRONMENT above), and may cause subshells to exit before executing all the commands in the subshell.

Only last command if executed in ||/&& chain can trigger exit.

The following expression will fail from the reason above.

true && false

but the following will not:

if true && false                                                                
then                                                                            
  true                                                                          
fi                   

because of true && false is part of test following if.

Answer 3

Regarding my problem of the script unexpectedly dying, it is due to running those short tests in a subshell, or from a function, and said function returns a value different than zero:

Original example:

$ set -e
$ false && true
$ echo $?
1

Using a subshell or function:

$ set -e
$ (false && true)
<script died>

$ set -e
$ variable=$(false && true)
<script died>

$ set -e
$ foo()(false && true)
$ foo
<script died>

$ set -e
$ foo(){ false && true; }
$ foo
<script died>

Possible solutions:

$ set -e
$ (false && true ||:)
$ (false && true) ||:
$ (if false; then true; fi)

Answer 4

The behaviour of set -e is unintuitive in all but the simplest cases and has changed several times over the years. Therefore I do not recommend using it except in really simple scripts (plain sequences of commands).

Note that commands in a Makefile are usually executed with set -e in effect, so it is still often needed to understand something about how it works.

The && and || operators allow a similar effect with relatively little clutter, but make it explicit. In particular, a && might be placed at the end of a line much like a `;'.