-1

A bash function, prepend_line, takes two parameters: a string and a fully-qualified path to a file. It's used for logging, inserting the current date/time and the string at the top of the log file.

Standalone use works fine: prepend_line "test string" "$log_file"

How can I get the output from a command, e.g. mv -fv "$fileOne" "$fileTwo" to be used as the first parameter for prepend_line?

I've tried various combinations of piping to xargs, but I don't understand how it works and I'm not convinced it's the best way in any case.

Lorccan
  • 793
  • 5
  • 20

1 Answers1

1

If you really have to:

export -f prepend_line
mv -fv "$fileOne" "$fileTwo" |
xargs -0 bash -c 'prepend_line "$1" "$log_file"' --

The -0 parses the line as beeing zero delimetered. As there should be no zeros in mv -v output, as filenames can't have a zero byte, you will get only a single element. This element/line will be passed as the first argument to the bash subshell.

Tested with:

prepend_line() {
  printf "%s\n" "$@" | xxd -p
}

fileOne=$'1\x01\x02\x031234566\n\t\e'
fileTwo=$'2\x01\x02\x031234566\n\t\e \n\n\n'
export -f prepend_line

printf "%s\n" "$fileOne -> $fileTwo" |
xargs -0 bash -c 'prepend_line "$1" "$log_file"' --

The script will output (output from the xxd -p inside prepend_line):

31010203313233343536360a091b202d3e2032010203313233343536360a
091b200a0a0a0a0a0a

Same hex output with some extra newlines and comments:

# first filename $'1\x01\x02\x031234566\n\t\e'
31010203313233343536360a091b
# the string: space + '->' + space
202d3e20
# second filename $'2\x01\x02\x031234566\n\t\e \n\n\n'
32010203313233343536360a091b200a0a0a0a0a0a

If you really have to parse some strange input's you can convert your string to hex with xxd -p. Then, later, convert it back to machine representation with xxd -r -p and streaming right into the output:

prepend_line() {
    # some work

    # append the output of the "$1" command to the log_file
    <<<"$1" xxd -p -r >> "$2"

    # some other work
}

prepend_line "$(mv -fv "$fileOne" "$fileTwo" | xxd -p)" "$log_file"

But I doubt you will ever need to handle such cases. Who names filenames using $'\x01' and suffixes with empty newlines 'great_script.sh'$'\n\n'?

Anyway, objectively I would rather see the interface as using a stream:

 mv -fv "$fileOne" "$fileTwo" | prepend_line "$log_file"

It needs set -o pipefail to propagate errors correctly. Inside prepend_line I would just redirect the output to the log file or some temporary file, sparing the need of parsing and corner cases.

KamilCuk
  • 120,984
  • 8
  • 59
  • 111
  • Wow! That's certainly comprehensive. Thank you. I'll take your advice and try looking at `read` to rewrite the passing of data to the function. (I assume that's what you're talking about?) – Lorccan Jun 26 '19 at 18:20
  • 1
    `read` bash builtin is extremely (very, very, very) slow. Don't use them, at least avoid using it. Only as a last resort or in small scripts. If `prepend_line` all does is prepends a line to a file, solve it using streams - pipes, command substitution, redirections - don't parse it. The whole function could look only like this: `prepend_line() { cat - "$2" | sponge "$2"; }`. Without `sponge` use temp file `prepend_line() { tmp=$(mktemp); cat - "$2" > "$tmp"; mv "$tmp" "$2"; }` The `cat - "$2"` will first read stdin, then the file, thus prepend stdin to the file. – KamilCuk Jun 26 '19 at 21:16
  • Thanks. I don't have sponge and don't really want to install something non-standard for portability reasons. I like `{ tmp=$(mktemp); cat - "$2" > "$tmp"; mv "$tmp" "$2"; }`. How would I be able to prefix the line with my date/time string (date '+%Y-%m-%d %H:%M:%S')? – Lorccan Jun 26 '19 at 22:34
  • 1
    if a single line I would just `sed 's/^/'"$(date +....)"'/' `, if more then `awk` script would be the fastest and most portable. There is also the `ts` utility from moreutils, I like it very much. `sponge` is also from moreutils. Also [this thread](https://stackoverflow.com/questions/6375654/how-to-add-date-string-to-each-line-of-a-continuously-written-log-file). – KamilCuk Jun 26 '19 at 22:46
  • Sorry, but you've lost me! Where would `sed` be used - in the function or to add to the stdout from `mv` before passing to the function? – Lorccan Jun 26 '19 at 22:57
  • 1
    In the function while streaming. `prepend_line() { tmp=$(mktemp); sed 's/^/'"$(date +%s)"'/' | cat - "$2" > "$tmp"; mv "$tmp" "$2"; }`. – KamilCuk Jun 26 '19 at 23:14
  • That all works perfectly - and I have learned something useful. Thank you again. – Lorccan Jun 27 '19 at 08:55