Substituting variables in a text string

Question

I have a text string in a variable in bash which looks like this:

filename1.txt
filename2.txt

varname1 = v1value
$(varname1)/filename3.txt
$(varname1)/filename4.txt

varname2 = $(varname1)/v2value
$(varname2)/filename5.txt
$(varname2)/filename6.txt

I want to substitute all of the variables in place, producing this:

filename1.txt
filename2.txt

v1value/filename3.txt
v1value/filename4.txt

v1value/v2value/filename5.txt
v1value/v2value/filename6.txt

Can anyone suggest a clean way to do this in the shell?

How large is this file? This is the kind of problem that's actually quicker to solve by hand, with a good text editor, unless the file is really large — salezica, Sep 04 '14 at 20:53
This is just a sample. There are actually several hundred files like this which each have between zero and 20 substitutions in them. — user2664470, Sep 04 '14 at 21:27

score 2 · Answer 1 · answered Sep 05 '14 at 02:00

In awk:

BEGIN {
    FS = "[[:space:]]*=[[:space:]]*"
}

NF > 1 {
    map[$1] = $2
    next;
}

function replace(     count)
{
    for (key in map) {
        count += gsub("\\$\\("key"\\)", map[key])
    }

    return count
}

{
    while (replace() > 0) {}
    print
}

In lua:

local map = {}

--for line in io.lines("file.in") do -- To read from a file.
for line in io.stdin:lines() do -- To read from standard input.
    local key, value = line:match("^(%w*)%s*=%s*(.*)$")
    if key then
        map[key] = value
    else
        local count
        while count ~= 0 do
            line, count = line:gsub("%$%(([^)]*)%)", map)
        end
        print(line)
    end
end

score 1 · Accepted Answer · answered Sep 04 '14 at 21:31

1

I found a reasonable solution using m4:

function make_substitutions() {
    # first all $(varname)s are replaced with ____varname____
    # then each assignment statement is replaced with an m4 define macro
    # finally this text is then passed through m4

    echo "$1" |\
    sed 's/\$(\([[:alnum:]][[:alnum:]]*\))/____\1____/' | \
    sed 's/ *\([[:alnum:]][[:alnum:]]*\) *= *\(..*\)/define(____\1____, \2)/' | \
    m4
}

answered Sep 04 '14 at 21:31

user2664470

781
1
7
17

You could use this same sort of pattern with any language or templating/macro system that allows that sort of runtime evaluation. You could even do it with the shell itself with some effort (and sanitization). An (single-pass probably) awk solution should also be fairly straightforward. – Etan Reisner Sep 04 '14 at 21:35

clt60 · Answer 3 · 2014-09-04T23:52:00.093

0

Perhaps

echo "$string" | perl -nlE 'm/(\w+)\s*=\s*(.*)(?{$h{$1}=$2})/&&next;while(m/\$\((\w+)\)/){$x=$1;s/\$\($x\)/$h{$x}/e};say$_'

prints

filename1.txt
filename2.txt

v1value/filename3.txt
v1value/filename4.txt

v1value/v2value/filename5.txt
v1value/v2value/filename6.txt

edited Sep 04 '14 at 23:52

answered Sep 04 '14 at 23:46

clt60

62,119
17
107
194

Substituting variables in a text string

3 Answers3