3

I have the following (simplified) code attempts in 2 shell scripts below.

The script calls an R script in which code is run that will generate standard output and error stream depending on what happens.

What I'm trying to achieve is to have both the output and error stream display on the console as usual but when the script is run and it fails (e.g. the RScript produces an error stream) I want to save that error message that R generates in a sqlite database but still have the normal output and errors display on the console as well when it exits. I have tried many forms of redirecting output of the Rscript function, but I either end up saving nothing to the database (apart from the line number) or I can save the error message but nothing will be put on the console...

This will not save error message to the DB (line number will), but everything will be on console

updateDBwhenError() {
    sqlite3 "myDB.db" "INSERT INTO logs VALUES('$1')"   
}

err_report() {
    #Tryign to capture both line error and message
    updateDBwhenError "Error Line $1 $2"
    exit
}
trap 'err_report ${LINENO}' ERR

Rscript testScript.R

This will save the error message to the DB, but nothing will be on console anymore

updateDBwhenError() {
    sqlite3 "myDB.db" "INSERT INTO logs VALUES('$1')"   
}

err_report() {
    #Tryign to capture both line error and message
    updateDBwhenError "Error Line $1 $rErr"
    exit
}
trap 'err_report ${LINENO}' ERR

rErr=$(Rscript testScript.R 2>&1)

I looked everywhere for a way to capture just the error stream to a variable and keep the output untouched on the console (both output and error), but I'm stuck.

Ian Campbell
  • 23,484
  • 14
  • 36
  • 57
pieterjanvc
  • 273
  • 2
  • 7
  • This [How can I capture STDERR into bash variable and not affect STDOUT?](https://stackoverflow.com/questions/48180205/how-can-i-capture-stderr-into-bash-variable-and-not-affect-stdout) not answering the question. – Ivan Sep 18 '20 at 11:27

2 Answers2

1

Try this

exec 5>&1
exec 6>&1
rErr=$(Rscript testScript.R 2>&1 1>&6 | tee /dev/fd/5)

Test script

$ cat test
#!/bin/bash
ls
ls sdfgds

Testing with this script

$ exec 5>&1
$ exec 6>&1
$ err=$(./test 2>&1 1>&6 | tee /dev/fd/5)
file  new_file  test  xml
ls: cannot access 'sdfgds': No such file or directory

$ echo "$err"
ls: cannot access 'sdfgds': No such file or directory
Ivan
  • 6,188
  • 1
  • 16
  • 23
0

Shortly

Using unnamed fifos (WARNING, this will work because OS do buffering, so only while output stay under 64Kb!), there is a subtle way:

exec {HOLDERR}<> <(:)
ls -ld /t{mp,nt} 2>&${HOLDERR}
read -t 0 -u $HOLDERR && read -ru $HOLDERR errmsg
exec {HOLDERR}<&-

This must output something like:

drwxrwxrwt 4 root root 4096 Jan 01 1970 /tmp

and popilate $errmsg with something like: declare -p errmsg

declare -a errmsg=([0]="ls: cannot access '/tnt': No such file or directory")

One step further

exec {HOLDERR}<> <(:)
ls -ld /t{mp,nt} 2>&${HOLDERR}
errmsg=()
while read -t 0 -u $HOLDERR;do
    read -ru $HOLDERR line
    errmsg+=("$line")
done
exec {HOLDERR}<&-
printf "%s\n" "${errmsg[@]}"

Full usable version

With trap SIGCHLD for flushing fifo.

printmsg() {
    local out=()
    out=("${errmsg[@]}")
    errmsg=("${errmsg[@]:${#out[@]}}")
    [ "${#out[@]}" -gt 0 ] &&
    printf "Err: %s\n" "${out[@]}"
}
checkmsg() {
    local line
    while read -u $HOLDERR -t 0 ;do
    read -ru $HOLDERR line &&
        errmsg+=("$line")
    done
}
msg=()
trap checkmsg CHLD
exec {HOLDERR}<> <(:)

Then

ls -ld /t{mp,nt} 2>&${HOLDERR}

must output something like:

drwxrwxrwt 4 root root 4096 Jan 01 1970 /tmp

followed by:

printmsg

will show

Err: ls: cannot access '/tnt': No such file or directory
F. Hauri - Give Up GitHub
  • 64,122
  • 17
  • 116
  • 137
  • `exec {HOLDERR}<> <(:)` It's worth noting that it depends on fifo buffering and that it can block. Fifo has it's own buffering equal to 65536 bytes (as I'm reading the [man page](https://man7.org/linux/man-pages/man7/pipe.7.html)), potentially it can block if a process would write more bytes, which would block your script. The solution here is multiprocessing - running `ls` as a background process and reading output while `wait`ing on it. – KamilCuk Sep 18 '20 at 08:28
  • 1
    @KamilCuk You're right! Answer edited! But of course, this could block, but not with raisonable `STDERR` output (smaller than 64k;). – F. Hauri - Give Up GitHub Sep 18 '20 at 08:33
  • Hi. Thanks for the response! I will look into this one but I admit it goes well beyond my current understanding so I'll have to work on it to see if I can get it implemented and working :) I thought I was just missing something easy, but it seems if I really want to pull this off I'll need a lot more complicated code... I'll keep you posted – pieterjanvc Sep 18 '20 at 10:53