Can't exit and strange order of execution with tee

Question

I am getting some strange behavior when piping the output of a function to the tee command. The first problem is that I can't exit the program when using the exit command when called from a function piped to tee. For Example:

myfunction(){
    # Some stuff here
    exit 1
}

myfunction | tee -a $UPGRADE_LOG

When I run the above code, the program fails to exit and runs to completion.

The other issue I am having is that tee seems to have cause some code to run in such a way that the sequential order is undone. I have the following output:

SHOWING SYSTEM-WIDE AND INSTATNCE MEMORY USAGE:
Are you sure you would like to back up the instance given current memory contraints? [y/n]: Filesystem                           Size   Used  Avail  Use%  Mounted on
/dev/mapper/system-root              15G    13G   1.5G   90%   /
Log File Size: 24K   Total Size to Package: 248K Available Space: 1.5G

When it should run as:

SHOWING SYSTEM-WIDE AND INSTATNCE MEMORY USAGE:
Filesystem                           Size   Used  Avail  Use%  Mounted on
/dev/mapper/system-root              15G    13G   1.5G   90%   /
Log File Size: 24K   Total Size to Package: 248K Available Space: 1.5G
Are you sure you would like to back up the instance given current memory contraints? [y/n]:

Things work properly when not using tee. The issues seem to be related to one another. Any ideas why this is the case and what I should do about it?

Please don't ask multiple questions instead split them. Note also that you increase your chances of getting good answers if you come as close as possible to a MCVE: https://stackoverflow.com/help/mcve — Micha Wiedenmann, Oct 16 '18 at 13:59
These questions seem to be linked to the same issue, so I put them together. Sorry if it brought about any confusion. — willh99, Oct 16 '18 at 14:02
No, the code doesn't *run* non-sequentially -- what happens is that your stdout and stderr are going through different processes so they arrive at different times. The original writes still happen at the same time, though. — Charles Duffy, Oct 16 '18 at 14:39
Yeah I knew the code ran sequentially. I was just pointing out that it *behaved* as such. I assumed this was because they were running separately, I just wasn't sure why. Thanks for the clarifications! — willh99, Oct 16 '18 at 14:44

Charles Duffy · Accepted Answer · 2018-10-16T14:48:47.930

kvantour's answer tells you why this happens. This one tells you how to fix it. :)

with_logs_piped() {
  local logfile=$1; shift
  "$@" > >(tee -a -- "$logfile") 2>&1  # 2>&1 redirects stderr through the same tee so it's
}                                      # ...also logged, and shows up sync'd with stdout.

myfunction() {
    # Some stuff here
    exit 1
}

with_logs_piped "$UPGRADE_LOG" myfunction

What's important here is that instead of using a regular pipeline, we're using a process substitution for tee -- so myfunction runs in your shell itself, not a subshell, so the exit applies properly.

As for why redirecting stdout through tee desynchronizes stderr, see Separately redirecting and recombining stderr/stdout without losing ordering

This solution is working perfectly. Thanks for the explanation as well! — willh99, Oct 16 '18 at 16:04

score 1 · Answer 2 · edited Jun 20 '20 at 09:12

1

The exit statement exits the (sub-)shell the process is running in. Now here comes the surprise:

Pipelines

A pipeline is a sequence of one or more commands separated by one of the control operators | or |&. The format for a pipeline is:
[time [-p]] [ ! ] command [ [|⎪|&] command2 ... ]
<snip>

Each command in a pipeline is executed as a separate process (i.e., in a subshell).

_{source: man bash}

So, the exit statement from the function just kills the subshell of the pipeline. This actually means that exit does nothing in a pipeline

$ exit | exit | echo foo
foo
$ exit | exit
$ # shell not terminated

Note: this is clearly shell dependent because zsh behaves differently.

edited Jun 20 '20 at 09:12

Community

1
1

answered Oct 16 '18 at 14:30

kvantour

25,269
4
47
72

1

"Does nothing" isn't *quite* accurate; `exit 2 | exit 3 | true` sets `${PIPESTATUS[0]}` to 2 and `${PIPESTATUS[1]}` to 3, and if `set -o pipefail` were active, would cause the pipeline as a whole to have an exit status of 3. Granted, though, that those effects aren't what the OP wanted. :) – Charles Duffy Oct 16 '18 at 14:45
@CharlesDuffy True. Also if the option `lastpipe` is set, it will have an effect on the last `exit` statement (in the eg. `exit | exit`) – kvantour Oct 16 '18 at 15:12

Can't exit and strange order of execution with tee

2 Answers2