Bash subshell input with variable number of subshells

Question

I want to grep lines from a variable number of log files and connect their outputs with paste. If I had a fixed number of outputs, I could do it thus:

paste <(grep $PATTERN $FILE1) <(grep $PATTERN $FILE2)

But is there a way to do this with a variable number of input files? I want to write a shell script whose arguments are the input files. The shell script should paste the grepped lines from ALL of them.

Grep will accept a list of files, perhaps a wildcard would do what you need? you can append to a file using the pipe append `>>` so delete the output, then run your greps in a loop each appending to the file? — Gem Taylor, Aug 06 '18 at 19:11

score 0 · Accepted Answer · answered Aug 06 '18 at 19:22

Use explicit named pipes, instead of process substitution.

pipes=()
for f in "$FILE1" "$FILE2" "$FILE3"; do
    n="$(mktemp)"  # Or some other command to create a temporary name
    mkfifo "$n"
    pipes+=( "$n" )
    grep "$PATTERN" "$f" > "$n" &
done

paste "${pipes[@]}"
rm  "${pipes[@]}"   # When done with them

score -1 · Answer 2 · answered Aug 06 '18 at 19:15

-1

You can do this by combining find command to list the files and piping its output to grep usings xargs to ensure grep is applied on each file listed in find command

$ find /dir/containing/files -name "file.*" | xargs grep $PATTERN

answered Aug 06 '18 at 19:15

rajeshnair

1,587
16
32

This doesn't use the output of each `grep` as a input file for `paste`, though. – chepner Aug 06 '18 at 19:18

Bash subshell input with variable number of subshells

2 Answers2