2

Below I have a script that is collecting the process ids of individual commands, and appending them to an array in bash. For some reason as you can see stdout below, the end resulting array just contains one item, the latest id. How can the resulting PROCESS_IDS array at the end of this script contain all four process ids?

PROCESS_IDS=()

function append {
    echo $1
    PROCESS_IDS=("${PROCESS_IDS[@]}" $1)
}

sleep 1 && echo 'one' & append $! &
sleep 5 && echo 'two' & append $! &
sleep 1 && echo 'three' & append $! &
sleep 5 && echo 'four' & append $!
wait

echo "${PROCESS_IDS[@]}"

Here is the stdout:

83873
83875
83879
83882
three
one
four
two
83882
ThomasReggi
  • 55,053
  • 85
  • 237
  • 424
  • 1
    Try removing the trailing `&` at the end of the first three lines. – Pavel Dec 08 '17 at 16:19
  • @pavel I want these four lines to run in parallel. – ThomasReggi Dec 08 '17 at 16:20
  • I just did ask you asked and it worked. However, I would like to run them in parallel, I still have access to the `$1` variable from within `append` it `echo`s properly it's just not storing to the array correctly. – ThomasReggi Dec 08 '17 at 16:21
  • 2
    `process_ids+=( "$!" )` is a considerably terser way of appending to an array. – Charles Duffy Dec 08 '17 at 16:30
  • 1
    @ThomasReggi If you remove the & from the end, the sleeps will still run in parallel. It is just the assignments to your array that run serially. – William Pursell Dec 08 '17 at 16:34
  • btw, using ksh-style function syntax in bash doesn't actually give you the advantages in bash it does in ksh -- so to someone coming from a ksh background (who expects `function foo {` to make variables local by default, for example), it's simply misleading. Using POSIX-compliant function syntax is a bit more clear, in terms of doing the same thing across all shells. – Charles Duffy Dec 08 '17 at 16:43

2 Answers2

3

Don't send the append operation itself to the background. Putting an & after the content you want to background but before the append suffices: The sleep and echo are still backgrounded, but the append is not.

process_ids=( )
append() { process_ids+=( "$1" ); }       # POSIX-standard function declaration syntax

{ sleep 1 && echo 'one'; } & append "$!"
{ sleep 5 && echo 'two'; } & append "$!"
{ sleep 1 && echo 'three'; } & append "$!"
{ sleep 5 && echo 'four'; } & append "$!"

echo "Background processes:"              # Demonstrate that our array was populated
printf ' - %s\n' "${process_ids[@]}"

wait
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • Although probably good style to include them, the braces are not necessary. `sleep 1 && echo one &` is equivalent to `{ sleep 1 && echo one; } &` From the bash manpage: `Of these list operators, && and || have equal precedence, followed by ; and &, which have equal precedence.` – William Pursell Dec 08 '17 at 16:36
  • 1
    @WilliamPursell, *nod* -- I'm falling back on "good style", here -- I find this to be much less ambiguous to a reader in terms of its behavior. – Charles Duffy Dec 08 '17 at 16:37
1

My guess is that whenever you send a function call to the background, it has a copy of the global variable on its own, so they're appending the PID to four independent copies of PROCESS_IDS. That's why every function call finds that it is empty and stores a single PID in it.

http://www.gnu.org/software/bash/manual/bashref.html#Lists

If a command is terminated by the control operator ‘&’, the shell executes the command asynchronously in a subshell. This is known as executing the command in the background.

If you want to collect the outputs from all four function calls, let them write to disk and read the output at the end:

function append {
    echo $1 | tee /tmp/file.$1
}

sleep 1 && echo 'one' & append $! &
sleep 5 && echo 'two' & append $! &
sleep 1 && echo 'three' & append $! &
sleep 5 && echo 'four' & append $!
wait

cat /tmp/file.*

Edit: this is just a proof of concept - don't do it with file system anyway (as William pointed out, this is going to be error prone unless you take care of uniqueness and synchronization). I only wanted to illustrate that you need to find another way of getting the information out of the subshells.

Pavel
  • 7,436
  • 2
  • 29
  • 42