3

In Linux I'm starting a program called $cmd in an init script (SysVInit). I'm already redirecting stdout and stderr of $cmd into two different logfiles called $stdout_log and $stderr_log. Now I also want to add a timestamp in front of every line printed into the logfiles.

I tried to write a function called log_pipe as follows:

log_pipe() {
    while read line; do
        echo [$(date +%Y-%m-%d\ %H:%M:%S)] "$line"
    done
}

then pipe the output of my script into this function and after that redirect them to the logfiles as follows:

$cmd | log_pipe >> "$stdout_log" 2>> "$stderr_log" &

What I get is an empty $stdout.log (stdout) what should be okay, because the $cmd normally doesn't print anything. And a $stderr.log file with only timestamps but without error texts.

Where is my faulty reasoning?

PS: Because the problem exists within an init script I only want to use basic shell commands and no extra packages.

Iniesta8
  • 399
  • 2
  • 15
  • Did you consider the 'ts' command at all? Candidly speaking, there's no need for the log_pipe() program as unix/linux already has the 'ts' command which does exactly what you're trying to do. do " $ man ts " to see if that satisfies you. e.g usage: "$ cmd 2>&1 | ts '[%Y-%m-%d %H:%M:%S %Z]' | tee /your-path/outputs.log > /dev/null 2>&1 &" – DanglingPointer Mar 21 '20 at 02:30

3 Answers3

4

In any POSIX shell, try:

{ cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log

Also, if you have GNU awk (sometimes called gawk), then log_pipe can be made simpler and faster:

log_pipe() {  awk '{print strftime("[%Y-%m-%d %H:%M:%S]"),$0}'; }

Example

As an example, let's create the command cmd:

cmd() { echo "This is out"; echo "This is err">&2; }

Now, let's run our command and look at the output files:

$ { cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log
$ cat stdout.log
[2019-07-04 23:42:20] This is out
$ cat stderr.log
[2019-07-04 23:42:20] This is err

The problem

cmd | log_pipe >> "$stdout_log" 2>> "$stderr_log"

The above redirects stdout from cmd to log_pipe. The stdout of log_pipe is redirected to $stdout_log and the stderr of log_pipe is redirected to $stderr_log. The problem is that the stderr of cmd is never redirected. It goes straight to the terminal.

As an example, consider this cmd:

cmd() { echo "This is out"; echo "This is err">&2; }

Now, let's run the command:

$ cmd | log_pipe >>stdout.log 2>>stderr.log
This is err

We can see that This is err is not sent to the file stderr.log. Instead, it appears on the terminal. It is never seen by log_pipe. stderr.log only captures error messages from log_pipe.

John1024
  • 109,961
  • 14
  • 137
  • 171
  • This works, but my command has to run in background because it is within an init script, therefore i have to do: ```bash ({ cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log) & echo $! > "$pid_file" ``` right? But I think in this case the pid in the $pid_file is not the pid of $cmd... – Iniesta8 Jul 05 '19 at 08:52
  • 1
    Yes, that's correct. Try `({ cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log) & pgrep cmd > "$pid_file"`. – John1024 Jul 05 '19 at 18:46
1

In Bash, you can also redirect to a subshell using process substitution:

logger.sh

#!/bin/bash
while read -r line; do
   echo "[$(date +%Y-%m-%d\ %H:%M:%S)] $line"
done

redirection

cmd > >(logger.sh > stdout.log) 2> >(logger.sh > stderr.log)
UtLox
  • 3,744
  • 2
  • 10
  • 13
0

This works, but my command has to run in background because it is within an init script, therefore i have to do:

({ cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log) &
echo $! > "$pid_file"

right?

But I think in this case the pid in the $pid_file is not the pid of $cmd...

Iniesta8
  • 399
  • 2
  • 15