Binary safe bash 'here string'?

Question

I'd like to save output of a command to a variable for later multiple uses. Bash provides Here String functionality for that purpose. However it is not binary safe. It sometimes adds new lines:

$ a=''
$ xxd <<< "$a"
00000000: 0a

Is there any binary safe alternative?

I use the variable in for loop so IIUIC it disqualifies tee command and any pipe like solution. I'd also prefer something else than temporary files as the are slow and clumsy to work with (require a writable directory, clean-up).

Bash variables aren't binary-safe to begin with (they can't contain `NUL`s, for instance), so as long as you're fetching the data from a variable, you have already disqualified it. Is it the newline specifically that you want to strip? — Dolda2000, Feb 24 '16 at 00:51
@Dolda2000 This is not true. Bash variables *can* contain `NUL` bytes. Try `a="a\0b"; echo -en "$a" | xxd` — hek2mgl, Feb 24 '16 at 00:56
@hek2mgl: Then please explain why the following command prints "6" rather than "7": `foo="$(base64 -d <<<"Zm9vAGJhcgo=")"; echo ${#foo}` — Dolda2000, Feb 24 '16 at 01:01
@hek2mgl: As for your `echo -en` example, the variable does not store any `NUL`, it simply stores `\0`, literally. It is `echo` that replaces that with a `NUL` since you gave it the `-e` option. — Dolda2000, Feb 24 '16 at 01:07
@Dolda2000 My problem is that `wc -l <<< "$a"` is not reliable. It returns one line even if the variable is empty. But I'm also curious what is a general solution. I don't need to use variables. If they are no suitable for that purpose, what is the alternative? — czerny, Feb 24 '16 at 07:47

score 1 · Answer 1 · answered Feb 24 '16 at 14:20

The answer depends on what, exactly, it is that you need. If your problem is only the newline that here-strings add, then all you need is echo -n:

$ foo=bar
$ echo -n "$foo" | od -t x1
0000000 62 61 72

If you need to preserve the trailing newline(s) that command-substitution strips, or you truly need full binary safety, however, then there are no "work-arounds", unfortunately. Command-substitution will always strip trailing newline no matter what, and as mentioned in the comments, shell variables are not binary safe as they cannot contain NULs. If you need any of those things, then I'm pretty sure your only option is using temporary files.

As for using temporary files, however, the problem you state of finding a writable directory should be a small one, as /tmp is always guaranteed to be writable by all unless you're working on a really weird system, or your script is supposed to run during an incomplete or failed boot, perhaps. In that case, you'll just have to switch to C instead. Otherwise, just use the mktemp command. As for cleanup, you may want to use the trap built-in command.

Binary safe bash 'here string'?

1 Answers1