Sed replace with the output of a bash command using capture group as argument

Question

I'm trying to make some base64 substitution with sed.

What I'm trying to do is this:

sed -i "s|\(some\)\(pattern\)|\1 $(echo "\2" | base64 -d)|g" myFile

In English that would be:

• Math a pattern
• Capture groups
• Use the captured group in a bash command
• Use the output of this command as a replacement string

So far my command doesn't work since \2 is known only by sed and not by the bash command I'm calling.

What elegant solution to I have to pass a capture group to a command of which I want to use the output?

---

Edit

Here is a minimal example of what I'm trying to do:

I have the following file:

someline
someline
Base64Expression stringValue="Zm9v"
someline
Base64Expression stringValue="YmFy"

And I want to replace the base64 by plain text:

someline
someline
Base64Expression stringValue="foo"
someline
Base64Expression stringValue="bar"

In the future I'll have to do the backward operation (encoding string in base64 on the decoded file)

I've started using awk but I though it could get simpler (and much more elegant) with sed. So far with awk I have this (where $bundle is the file I'm editing):

#For each line containing "Base64Expression"
#Put in the array $substitutions[]:
# The number of the line (NR)
# The encoded expression ($2)
# The decoded expression (x)
substitutions=($(awk -v bd=$bundle '
    BEGIN {
        # Change the separator from default
        FS="""
        ORS=","
        OFS=","
    }
    /Base64Expression/ {
        #Decode the base64 lines
        cmd="echo -ne \""$2"\" | base64 -d"
        cmd | getline x

        if ( (cmd | getline) == 0 ){
            print NR, $2, x
        }
    }
' $bundle))

# Substitute the encoded expressions by the decoded ones
# Use the entries of the array 3 by 3
# Create a sed command which takes the lines numbers
for ((i=0; i<${#substitutions[@]}; i+=3))
do
    # Do the substitution only if the string is not empty
    # Allows to handle properly the empty variables
    if [ ${substitutions[$((i+1))]} ]
    then
        sed -i -e "${substitutions[$i]}s#${substitutions[$((i+1))]}#${substitutions[$((i+2))]}#" $bundle
    fi
done

Answer 1

You can use e in GNU sed to pass the substitution string to a shell for evaluation. This way, you can say:

printf "%s %s" "something" "\1"

Where \1 holds a captured group. All together:

$ sed -r 's#match_([0-9]*).*#printf "%s %s" "something" "\1"#e' <<< "match_555 hello"
something 555

This comes handy when you want to perform some shell action with a captured group, like in this case.

So, let's capture the first part of the line, then the part that needs to be encoded and finally the rest. Once this is done, let's print those pieces back with printf triggering the usage of base64 -d against the second slice:

$ sed -r '/^Base64/s#(.*;)([^\&]*)(&.*)# printf "%s%s%s" "\1" $(echo "\2" | base64 -d) "\3";#e' file
someline
someline
Base64Expression stringValue="foo"
someline
Base64Expression stringValue="bar"

Step by step:

sed -r '/^Base64/s#(.*;)([^\&]*)(&.*)# printf "%s%s%s" "\1" $(echo "\2" | base64 -d) "\3";#e' file
#        ^^^^^^^    ^^^  ^^^^^^  ^^^                        ^^^^^^^^^^^^^^^^^^^^^^^^       ^
#           |   first part  |   the rest                encode the 2nd captured group      |
#           |               |                                                              |
#           |           important part                                      execute the command
#           |
# on lines starting with Base64, do...

The idea comes from this superb answer by anubhava on How to change date format in sed?.

Answer 2

It sounds like this is what you're trying to do:

$ cat tst.awk
BEGIN { FS=OFS=""" }
/^Base64Expression/ {
    cmd="echo -ne \""$2"\" | base64 -d"
    if ( (cmd | getline x) > 0 ) {
        $2 = x
    }
    close(cmd)
}
{ print }

$ awk -f tst.awk file
someline
someline
Base64Expression stringValue="foo"
someline
Base64Expression stringValue="bar"

assuming your echo | base64 is the right approach.