In bash how to parse and/or redirect stdout for an arbitrary program I am backgrounding

Question

In a previous question I asked how to write to a program's stdin. This question builds off of that one. Now I am asking "how do I parse my called program's output".

The fact that I'm calling a program I can edit (Sum.sh) in this example is irrelevant to the question. I want this behavior in the event that I do not have control over the implementation of the program I am calling. I want to "replace" the physical human that a program expects and drive it via a script.

Given two scripts
Sum.sh:

#!/bin/bash 

sum=0
inp=0

while true;
do 
   printf "Sum: %d\n" $sum 
   printf "Give me an integer: "
   read inp 
   if [ "$inp" == 0 ]; then
      exit
   fi
   sum=$((sum+inp))
done

And driver.sh:

#!/bin/bash 
rm -f /tmp/sumpipe
mkfifo /tmp/sumpipe 

./Sum.sh < /tmp/sumpipe &


echo 2 > /tmp/sumpipe
echo 5 > /tmp/sumpipe 
echo 0 > /tmp/sumpipe 

sleep 1 # this is required for some reason or the script hangs

And I run driver.sh I get the following:

$ ./driver.sh 
Sum: 0
Give me an integer: Sum: 2
Give me an integer: Sum: 7
Give me an integer: $

The formatting of the terminal output is not important to this question.

What I want to do is parse said output in order to make decisions in driver.sh. In the following psuedo code of driver.sh I try to show what I want to do:

#!/bin/bash 
rm -f /tmp/sumpipe
mkfifo /tmp/sumpipe 

output=$(./Sum.sh < /tmp/sumpipe &)

while true; do
    if [ "$output" != empty ]; then
    (make some decision based on what output is and echo back to my process if necessary)
    fi
done

Unfortunately, $output never contains anything, and I have tried many versions of abusing read and piping and none of them have worked for me in this application. How can I redirect my backgrounded program's stdout so I can parse it?

I attempted OrangesV's proposed solution (and with deference to William Pursell's suggestion about the out buffer being too empty) with the following:

#!/bin/bash 
rm -f /tmp/sumpipe
mkfifo /tmp/sumpipe 

while read output; do 
   echo "2347823" > /tmp/sumpipe
   if [ -z $output ]; then 
      continue
   else
      echo "got it!"
      echo $output
      sleep 1
   fi
done < <(./Sum.sh < /tmp/sumpipe &)

Unfortunately, "got it!" is never echoed, so it seems it didn't work.

In case it's relevant, my bash version:

$ bash --version
GNU bash, version 4.4.0(7)-rc2 (x86_64-unknown-linux-gnu)

There is now a partial solution. Diego Torres Milano's answer does work. However, my stdio is being buffered so that I won't get anything till stdio has at least 4096 bytes in it. I can force output with the following version of driver.sh:

#!/bin/bash 

inpipe=/tmp/sumpipei
outpipe=/tmp/sumpipeo

rm -f $inpipe
rm -f $outpipe

mkfifo $inpipe
mkfifo $outpipe

./Sum.sh < $inpipe > $outpipe &


count=0
while true;
do
   while read line
   do
      echo "line=$line"
   done < $outpipe &
   echo 2 > $inpipe
done

I tried all sorts of invocations of stdbuf (like: stdbuf -o0 ./Sum.sh <$inpipe > $outpipe &). stdbuf does not apparently work in this situation to force off buffering because, according to the GNU coreutils manual on stdbuf for command in stdbuf option… command:

command must start with the name of a program that does ... not adjust the buffering of its standard streams (note the program tee is not in this category).

Which is unfortunately exactly what I'm doing here.

I'm marking Diego's answer as correct, even though it doesn't work until the stdio buffer issue is fixed. I'll make my next question to address the stdio buffering problem. I can almost taste the prize here.

The backgrounded process is not going to print anything until it has a lot to print (eg, 4096 characters). You've got a buffering problem. — William Pursell, Sep 28 '16 at 21:27
@WilliamPursell Is it possible to flush the process's stdout buffer? — durandaltheta, Sep 28 '16 at 21:31
@WilliamPursell If that's the case, can you explain to me why my attempt to capture said output is blank while my terminal fills with text normally when running the initial example scripts? I do not clearly understand the difference — durandaltheta, Sep 28 '16 at 22:02
When output is to a tty, a process will use line buffering by default. When it is to a pipe (as it is when doing process substitution) it will be fully buffered. To make it line buffered, you might try using `stdbuf`. — William Pursell, Sep 28 '16 at 22:12
@WilliamPursell So, perhaps `output=$(stdbuf -oL ./Sum.sh < /tmp/sumpipe &)` ? — durandaltheta, Sep 28 '16 at 23:21
what about another `mkfifo` to handle the output of `sum.sh` which would be read by `driver.sh` ? Good luck. — shellter, Sep 28 '16 at 23:23
@WilliamPursell Looks like standard usage of stdbuf won't work for this use case. In the stdbuf gnu [manual](https://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html) the "command" in `stdbuf option... command` "does not adjust the buffering of its standard streams (note the program tee is not in this category)." Updated question with more information — durandaltheta, Sep 29 '16 at 19:57

OrangesV · Answer 1 · 2016-09-28T22:13:35.893

0

You can do a while read within your driver.sh.

while read output; do
    if [ "${output}" == "something" ]; then
        #do something
    fi
done < <(./Sum.sh < /tmp/sumpipe &)

EDIT

This loop will not start iterating until the script's stdin receives input. So, in another shell/process an echo # > /tmp/sumpipe.

edited Sep 28 '16 at 22:13

answered Sep 28 '16 at 21:41

OrangesV

322
1
2
9

This doesn't seem to work, and have updated my question with my test code and more information – durandaltheta Sep 28 '16 at 21:59
@durandaltheta I see that you echoed a number to the pipe within the loop. In my answer I was assuming you would be echoing a value from a different shell/process. – OrangesV Sep 28 '16 at 22:07

score 0 · Accepted Answer · answered Sep 29 '16 at 03:29

0

Use another pipe for the output

mkfifo /tmp/output
./Sum.sh < /tmp/sumpipe > /tmp/output &

and you can process it with

while read line
do
    echo "line=$line"
done < /tmp/output &

answered Sep 29 '16 at 03:29

Diego Torres Milano

65,697
9
111
134

This actually works, but it doesn't fix an issue I'm running into with stdio buffering (as originally hinted by @WilliamPursell). I have to cause Sum.sh to spit out enough text (over 4096bytes) by spamming it with input over /tmp/sumpipe before anything appears – durandaltheta Sep 29 '16 at 19:59

In bash how to parse and/or redirect stdout for an arbitrary program I am backgrounding

2 Answers2

Linked