Add prefix in bash command output

Question

I would like to add a prefix in each new line of my command output.

I would like to do this because I will run multiple commands in parralel whiches will log in same output log.

I tried to do this with AWK without success

runcommand1 | "[prefix1]" + log > logfile &
runcommand2 | "[prefix2]" + log > logfile &

The result log would be

[prefix1] bla bla
[prefix2] blo blo blo
[prefix1] other bla
[prefix1] other bla
[prefix2] other blo blo

Is this different programs trying to write one regular file at the same time? I think you should serialize the logging task so that there is no possibility of clobbered output. Perhaps use different log files for different processes. See http://stackoverflow.com/questions/15384331/unix-can-i-write-to-the-same-file-in-parallel-without-missing-entries — pii_ke, Feb 27 '17 at 10:13

Akshay Hegde · Accepted Answer · 2017-02-27T13:23:52.340

9

I would like to add a prefix in each new line of my command output.

runcommand1 | "[prefix1]" + log > logfile &
runcommand2 | "[prefix2]" + log > logfile &
                                ^
    Second one will overwrite first opened logfile
    you have to append ">>" second one

Below mentioned few commands may help you.

You can use sed

yourcommand1 | sed  's/^/[prefix1] /' > logfile &

You can use awk

yourcommand1 | awk '{ print "[prefix1]", $0 }' > logfile &
yourcommand2 | awk '{ print "[prefix2]", $0 }' >> logfile &

if you want to pass prefix string to awk then

yourcommand1 | awk -v prefix="[prefix1]" '{ print prefix, $0 }' > logfile &
yourcommand2 | awk -v prefix="[prefix2]" '{ print prefix, $0 }' >> logfile &

OR

mystring1="[prefix1]"
mystring2="[prefix2]"
yourcommand1 | awk -v prefix="$my_string1" '{ print prefix, $0 }' > logfile &
yourcommand2 | awk -v prefix="$my_string2" '{ print prefix, $0 }' >> logfile &

If that's not all you need then update your question to show some more truly representative sample input (which you receive after running runcommand1/runcommand2) and expected output.

edited Feb 27 '17 at 13:23

answered Feb 27 '17 at 13:04

Akshay Hegde

16,536
2
22
36

1

Probably it's worth to mention io buffering and the `stdbuf` command. – hek2mgl Feb 27 '17 at 13:12
@hek2mgl: Yes true, most of the time we have to assume input could be like this :) – Akshay Hegde Feb 27 '17 at 13:15
1

Two different pipes writing to the same logfile? Won't the second "open" overwrite what the first "open" already wrote? This looks like a race-condition to me. – Jens Feb 27 '17 at 13:15
@jens true, I appended second one atleast to make answer meaningful, don't know what user is trying to do – Akshay Hegde Feb 27 '17 at 13:17
1

@AkshayHegde I will run two Docker containers and concat their ouputs – Lombric Feb 27 '17 at 15:22

Sergii Shymko · Answer 2 · 2021-01-07T23:53:00.050

2

The simplest way to prepend a prefix to each line of output (or append a suffix) is via printf:

runcommand1 | xargs printf -n 1 '[prefix]%s\n' > logfile &

This can be simplified even further if printing a space after the prefix is desirable:

runcommand1 | xargs echo -n 1 '[prefix]' > logfile &

Note that -n 1 is needed to live stream output line by line instead of buffering all lines.

edited Jan 07 '21 at 23:53

answered Dec 13 '20 at 03:43

Sergii Shymko

116
4

score 0 · Answer 3 · answered Mar 22 '23 at 00:55

I needed to add a prefix while streaming the output of multiple commands. In Bash I ended up doing this:

while read -r line; do echo "[$prefix] $line"; done < <({
  command1
  command2
  command3
})

Note that this only prefixes stdout and not stderr and it's also using bashisms.

Eduardo Cuomo · Answer 4 · 2023-04-29T20:56:04.037

Prefix with timestamp:

#!/usr/bin/env bash

{
  echo foo 1
  sleep 1
  echo bar 1 >&2
  echo foo 2
  sleep 1
  echo bar 2 >&2
} \
  2> >(sed "s/^/$(date '+%Y-%m-%d %H:%M:%S') [ERR] /" >&2) \
  > >(sed "s/^/$(date '+%Y-%m-%d %H:%M:%S') [INF] /")

Or, as a function/script:

#!/usr/bin/env bash

output-prefix() {
  "$@" \
    2> >(sed "s/^/$(date '+%Y-%m-%d %H:%M:%S') [ERR]${prefix} /" >&2) \
    > >(sed "s/^/$(date '+%Y-%m-%d %H:%M:%S') [INF]${prefix} /")
}

output-prefix ./script1.sh foo bar
# 2023-04-29 17:37:12 [INF] foo
# 2023-04-29 17:37:12 [ERR] test as errout
# 2023-04-29 17:37:12 [INF] bar

Source: https://gist.github.com/reduardo7/5d8443130495d3ad0e630eb344a37707

Add prefix in bash command output

4 Answers4