Bash - how to replace a single line starting with a pattern with multiple lines defined in a variable?

Question

I have a file that at some point contains a line like the following:

VIRTUAL_ENV="/afs/sern.ch/user/l/lronhubbard/virtual_environment"

It is the sole line in the file that begins with VIRTUAL_ENV. Using a script, I want to replace this line with the following lines:

directory_bin="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
directory_env="$(dirname "${directory_bin}")"
VIRTUAL_ENV="${directory_env}"

How could this be done?

To start with, I've got the replacement lines stored in a here document in the script:

IFS= read -d '' text << "EOF"
directory_bin="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
directory_env="$(dirname "${directory_bin}")"
VIRTUAL_ENV="${directory_env}"
EOF

Next, I can replace the line in the file with something else (here, xxx) using sed:

sed -i "s/^VIRTUAL_ENV.*/xxx/g" test.txt

Now, how can I use the defined here document variable ${text}, with all of its newlines and whatnot, instead of xxx in the sed command?

EDIT: Following the suggestion by rslemos, I have implemented the following using a temporary file instead of a here document:

#!/bin/bash

temporary_filename="$(tempfile)"

cat > "${temporary_filename}" << "EOF"
directory_bin="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
directory_env="$(dirname "${directory_bin}")"
VIRTUAL_ENV="${directory_env}"
EOF

sed -i "/^VIRTUAL_ENV.*/ {
   r ${temporary_filename}
   d
}" test.txt

rm "${temporary_filename}"

I'd still like to know how to use a here document directly so that I'm not unnecessarily using the hard drive.

Please don't post an answer as part of the question. If you want to add an answer of your own, post it as an answer instead. — tripleee, Jan 13 '16 at 12:39

rslemos · Accepted Answer · 2016-01-13T12:53:52.530

I would go with sed alone:

/^VIRTUAL_ENV/{
r source.txt
d
}

Where source.txt is the file where lie the new lines to replace a line starting with "VIRTUAL_ENV".

If you really need the new lines (source.txt) to be a here document, the /dev/fd/0 trick will work (in Linux at least). But you could also combine mktemp and mkfifo.

EDIT

To address the "I'd still like to know how to use a here document directly":

!/bin/bash

sed -e '/^VIRTUAL_ENV/{' -e 'r /dev/fd/0' -ed -e'}' test.txt << "EOF"
directory_bin="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
directory_env="$(dirname "${directory_bin}")"
VIRTUAL_ENV="${directory_env}"
EOF

Not very portable, because of the /dev/fd/0 part (works in Linux, though).

score 1 · Answer 2 · answered Jan 13 '16 at 15:09

This might work for you (GNU sed & bash):

cat <<\! | sed $'/PATTERN/{r /dev/stdin\n;d}' file
HERE DOCUMENT
STUFF
HERE
!

Use cat to pipe the stdin into the sed command and read it as a file using r command.

N.B. \! quotes all text in the here document use ! to interpolate variables. $'...' allows commands needing a newline (i.e. sed commands such as r,a,i,c,R,w,W) to be written as one line.

score 0 · Answer 3 · answered Jan 13 '16 at 12:37

Many dialects of sed allow you to escape newlines in the substitution:

sed -i 's/^VIRTUAL_ENV.*/directory_bin="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"\
directory_env="$(dirname "${directory_bin}")"\
VIRTUAL_ENV="${directory_env}"/' test.txt

That's a single-quoted string spanning three lines, with a backslash added for continuation at each internal line boundary.

This is not portable to every sed variant out there, but works fine on *BSD (including OSX) and Linux at least.

Incidentally, you could also simply put all of these commands on a single line, with semicolons between them.

The `/g` flag was clearly useless here, so I took it out. – tripleee Jan 13 '16 at 12:38 — tripleee, Jan 13 '16 at 12:38

Bash - how to replace a single line starting with a pattern with multiple lines defined in a variable?

3 Answers3