How to stop redirecting stdout when one program fails in the chain?

Question

In the normal case, this would output file.out

$ program1 | program2 | program3 > file.out

But what happens if program1 fails? What happens is that the rest of the chain still fires and the file is created

$ false | echo worked > file.out
$ cat file.out
worked

I don't want the file to appear.

There is another SO article about this that suggests adding an OR clause, like so:

$ false | echo worked > file.out || rm file.out
ls file.out
file.out  

This doesn't work. The second command fires and counts as a success.

That same SO article also suggests using a double ampersand, like so:

$ program1 && program2 && program2 > file.out

That doesn't work entirely. In my case program2 is expecting output from the stdout, so this chain ends up hanging because the chain is not stopping. You can see how this works with a trivial example:

$ echo something > stuff.txt
$ cat stuff.txt && false > file.out
something
$ cat file.out

It's not redirecting the output.

But, even worse, when things are happy, it doesn't work either.

$ echo something > stuff.txt
$ cat stuff.txt && true > file.out
something
$ cat file.out

In this case, file.out is created and it's blank. Uh-Oh.

---

I accepted an answer below. Calling set -o pipefail was the tip that I needed. In my real situation, I'm using a Makefile. To adapt this to work in a Makefile, I added this to the top of the file

SHELL=/bin/bash

And then to my target:

target:
    @set -o pipefail; program1 | program2 | program3 > $@

Answer 1

When bash executes program1 | program2 | program3 > file.out, it creates file.out before program1 is started. If you want to ensure that it is never created, you'll need to buffer the output (either in memory or in a temporary file). I find the cleanest syntax for that is something like:

if v=$( set -o pipefail; program1 | program2 | program3 ); then 
    echo "$v" > file.out
fi

or (this has different semantics, ignoring the return value. Depending on your use case, this may be acceptable):

v=$( program1 | program2 | program3 )
test -n "$v" && echo "$v" > file.out

If you're okay with creating the file and then deleting it, you can do

set -o pipefail
program1 | progam2 | program3 > file.out || rm file.out

If you don't want to use pipefail (eg, because you want the script to be portable), you can do something like:

{ { { program1 || echo . >&3; } | { program2 || echo . >&3;} | 
{ program3 || echo . >&3; } } 3>&1 >&4 | 
if grep -q .; then exit 1; else exit 0; fi ; } 4>&1;

Answer 2

false | echo worked > file.out || rm file.out

Above is the closest to what you want; if you turn on the pipefail option, it'll work. Like

$ set -o pipefail
$ false | echo foo > file || rm -f file
$ ls file
ls: cannot access 'file': No such file or directory

Concerning the exit status of a pipeline, and pipefail's effect on it, the manual says

The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.

Answer 3

Wrap your commands in functions and you may add any error handling you want:

f_program1()
{
  program1 || pkill -f program2
}

f_program2()
{
  program2 || pkill -f program3
}

f_program3()
{
  program3
}

f_program1 | f_program2 | f_program3 > file.out

I'm not sure that pkill is your best error handling strategy, so treat it as a placeholder for the sake of the argument that functions can make your troubles go away.