Streaming command substitution output to terminal w/o squelching color

Question

Is there a way to show the output of a command substitution, in color, while it is executing, while still storing it to a variable? This would mainly be for testing/debugging purposes, and the output would not be shown normally.

If I do this:

output=$(some_command)
if [ "$debug" ]; then
  echo "$output"
fi

...then this is close, but not what I want in a few ways:

Even if the command would have otherwise emitted output to the terminal in color, it's printed in black-and-white.
The output is only shown after the command is finished, rather than streaming as it runs.

How can I conditionally stream output to a terminal without squelching color?

`${some_command}` is a variable that you haven't set. You have set `command` but not used it anywhere... — 123, Nov 17 '15 at 15:48
...as for `declare command="some_command"`, that's Just Wrong. Read BashFAQ #50: http://mywiki.wooledge.org/BashFAQ/050 -- also, don't use the name `command`, which is also the name of a shell builtin. — Charles Duffy, Nov 17 '15 at 16:05
Sorry, I meant to use `command` in the later lines, not `some_command`. Fixed. I will try to clarify the rest of the question as best I can. — trysis, Nov 17 '15 at 16:08
Also, deciding whether a command failed by looking at its output is... not the usual way to do it; convention is to look at exit status instead. `if ! output=$(some_command); then echo "some_command failed!" >&2; fi` will both store the output and check the exit status. — Charles Duffy, Nov 17 '15 at 16:10
As for color, though, there simply is no generic way to do it, absent a tool like `expect` which can simulate a TTY: By convention, programs look at whether their output is direct to a TTY and turn color off if it's not; also, by convention, they typically provide some way to override that and force color anyhow. You'll need to look at usage specific to the single command you're running. — Charles Duffy, Nov 17 '15 at 16:11
@CharlesDuffy, I only wanted to store the output for debugging purposes. I could have used `$?` instead. I only wanted to use command substitution so I could see the output when debugging, but not all the time. — trysis, Nov 17 '15 at 16:18
@trysis, actually, if you're using `if` correctly, you typically don't need `$?` at all -- you can test the exit status directly rather than storing and then testing it later. — Charles Duffy, Nov 17 '15 at 16:18
@trysis, ...that said, if you want to only conditionally show output, I'd do that by redirecting a file descriptor to the console if debugging is enabled, or to /dev/null if it's not, and copying content you only optionally want to see to that FD. — Charles Duffy, Nov 17 '15 at 16:19
@CharlesDuffy, I think this is what I ultimately want to do. That said, this question doesn't really make sense, as you said. I'm thinking about deleting it, although that would remove all the upvotes/rep for your wonderful answer. — trysis, Nov 17 '15 at 16:27
@trysis, I'm willing to try my hand at rewriting the question to be more clear. :) — Charles Duffy, Nov 17 '15 at 16:28
Yes, thank you, you're amazing. I just feel bad all the rep from the question will go to me, not you, although I guess you'll get enough from your also-amazing answer. — trysis, Nov 17 '15 at 16:33
I'm thinking I may want an actual logger, like in [this question](http://unix.stackexchange.com/questions/106776/how-can-i-print-only-certain-commands-from-a-bash-script-as-they-are-run). There are just too many edge cases, and the script is slowly getting complicated. — trysis, Nov 17 '15 at 16:35
No worries; the real goal here is to build a good Q&A knowledgebase; the rep points are just keeping score. :) — Charles Duffy, Nov 17 '15 at 16:35
BTW, note the `set -o pipefail` in the long form of my answer -- really, you want that even if you're using the first short example, because it ensures that if `some_command` fails, the pipeline's overall exit status is that of the first failure rather than that of `tee`. — Charles Duffy, Nov 17 '15 at 16:36
You are right about `pipefail`. That really should be the default behavior. It seems to be similar to `try...catch` or even promises in some higher-order languages like JavaScript, though without the `catch` — trysis, Nov 17 '15 at 18:34
@trysis, the thing about shells is that there's a legacy of backwards compatibility going literally as far back as the 1970s. Sure, there are things we'd do differently if we were building a new shell today that didn't need to run old code, but... follow that path too far and you get zsh (which does the right thing rather than the compatible thing whenever there's a choice, and in consequence people who are overaccustomed to using it write code that's a huge mess of subtle bugs when targeting anything else because they're not used to having to think about the gotchas). — Charles Duffy, Nov 17 '15 at 23:59
@trysis, ...I'd actually argue that it's better to break with POSIX sh compatibility altogether (as `fish` does) if you want to go that route, rather than writing a language that's subtly incompatible but similar enough that habits can leak through without thinking about it. — Charles Duffy, Nov 18 '15 at 00:02
I understand, and I completely agree. I have been using `fish` recently, at home because at work we are using an extremely old version of RedHat. I was noting that it would make more sense to have `pipefail` as the default, but you are right that there is too much backwards compatibility to add it to `bash` now. — trysis, Nov 18 '15 at 01:53

Charles Duffy · Accepted Answer · 2015-11-17T16:26:46.993

To store a command's results to a variable as well as streaming its output to the console:

var=$(some_command | tee /dev/stderr)

If you want to force your command to think it's outputting directly to a TTY, and thus to enable color output if it would do so when not in a pipeline, use the tool unbuffer, shipped with expect:

var=$(unbuffer some_command | tee /dev/stderr)

All that said: If you only want to show debugging conditionally for a long script, it makes sense to put that conditional up front at the top of your script rather than scattering it around everywhere. For instance:

# put this once at the top of your script
set -o pipefail
if [[ $debug ]]; then
  exec 3>/dev/stderr
else
  exec 3>/dev/null
fi

# define a function that prepends unbuffer only if debugging is enabled
maybe_unbuffer() {
  if [[ $debug ]]; then
    unbuffer "$@"
  else
    "$@"
  fi
}

# if debugging is enabled, pipe through tee; otherwise, use cat
capture() {
  if [[ $debug ]]; then
    tee >(cat >&3)
  else
    cat
  fi
}

# ...and thereafter, when you want to hide a command's output unless debug is enabled:
some_command >&3

# ...or, to capture its output while still logging to stderr without squelching color...
var=$(maybe_unbuffer some_command | capture)

I was going to suggest something similar, `$(some_command | tee >(cat > /dev/stderr))`, which will work even if `/dev/stderr` isn't in the local file system. It has the same TTY-based issue, though. — chepner, Nov 17 '15 at 16:03
@chepner, good call; I incorporated that into the long-form extended answer — Charles Duffy, Nov 17 '15 at 16:45

Streaming command substitution output to terminal w/o squelching color

1 Answers1