7

I've got the following command:

$ cmd-a | while read -r line; do echo "${line}"; cmd-b; done

This works well, and will for all intents and purposes look like cmd-a is just printing its stdout as normal, but for each line we execute cmd-b as well.

Is there a cleaner way to do this?

cmd-a | xargs -n1 cmd-b would be nice, but it splits on all whitespace (I know GNU xargs has the -d option, but it's unfortunately not available to me,) and would suppress the output of cmd-a.

Marcus Stade
  • 4,724
  • 3
  • 33
  • 54
  • @JohnKugelman good point, thanks! You're right, I *do* want the output of `cmd-a` to be written to stdout as well as the output of `cmd-b`. I'll update the question to be clearer. – Marcus Stade Nov 15 '16 at 15:43
  • @anubhava not sure what you mean. I know what process substitution is, but how would you apply it in this case? – Marcus Stade Nov 15 '16 at 15:44
  • `while IFS= read -r line; do echo "${line}"; cmd-b; done < <(cmd-a)` – anubhava Nov 15 '16 at 15:45
  • 3
    This is perfectly clean, idiomatic code, [in line with best practices](http://mywiki.wooledge.org/BashFAQ/001). Why do you want something different? – Charles Duffy Nov 15 '16 at 15:48
  • Right @anubhava I see what you mean now, but that doesn't execute `cmd-b` (I'm guessing this is just an oversight) and isn't really easier to read *or* write. (Perhaps I should update the question to say "Is there a way to do this, which is less cumbersome to read and write?" rather than the "cleaner" comment?) – Marcus Stade Nov 15 '16 at 15:48
  • @CharlesDuffy right, "clean" isn't perhaps the right term to use. Really what I'm looking for is something more terse but without losing the readability of it, if that makes more sense? – Marcus Stade Nov 15 '16 at 15:49
  • 1
    Ahh. Terseness is the enemy of correctness in bash -- I strongly advise against making it a priority. – Charles Duffy Nov 15 '16 at 15:50
  • 1
    BTW, anubhava's suggestion means that changes to shell state (variables, ets) made in your loop are visible to the outer shell, so it *is* worth considering. See also [BashFAQ #24](http://mywiki.wooledge.org/BashFAQ/024). – Charles Duffy Nov 15 '16 at 15:51
  • @CharlesDuffy I don't necessarily disagree, but this is turning out to be a very common pattern for us and I guess before I write a utility for it I figured I'd check with the hive mind if there's a "better" way. Much obliged for you commentary and the BashFAQ link – hadn't seen it before! – Marcus Stade Nov 15 '16 at 15:52

1 Answers1

9

The existing proposal is entirely idiomatic and clean; BashFAQ #001 advises precisely the same mechanism to iterate over input line-by-line. There's no reason to replace it, but if you're using it frequently, it may make sense to encapsulate it.

Consider the following function:

for_each_line() {
  while IFS= read -r line; do
    printf '%s\n' "$line"
    line=$line "$@"
  done
}

...thereafter run as:

cmd-a | for_each_line cmd-b

...or, if your cmd-b is a function and you want it to be able to modify shell state, the above can be invoked as follows (per BashFAQ #24):

for_each_line cmd-b < <(cmd-a)

...which will make the preceding line available to cmd-b in the environment, should it have any reason to modify its behavior based on the value of same.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • Solid answer – much obliged! – Marcus Stade Nov 15 '16 at 15:57
  • 1
    One bug -- my original code was using `echo` rather than `printf`. That means that with bash running stock options, a line containing only `-n` would disappear; that a bash with the `xpg_echo` option enabled could behave differently; and that on other POSIX-y shells, [behavior was extremely poorly-defined](http://pubs.opengroup.org/onlinepubs/009695399/utilities/echo.html) -- see in particular the APPLICATION USAGE section of the linked standards document. Amended appropriately. – Charles Duffy Nov 15 '16 at 16:01