6

How do you completely escape the REPLACEMENT part of a perl subsitution with the s// operator? \Q and \E do not work as, shown below:

For context, this comes up when using perl to do big recursive search and replace operations driven from a bash script. This is not an easily avoidable situation.

Take this script for example:

$ cat example.sh
#!/bin/bash
set -v -x
EMAIL=user@example.org
echo "EMAIL = $EMAIL"
echo "Email address: here" | perl -p -e "s/here/$EMAIL/"
echo "Email address: here" | perl -p -e "s/here/\\Q$EMAIL\\E/"
echo "Email address: here" | perl -p -e "s/here/${EMAIL/@/\\@}/"

Let's run it:

$ ./example.sh
EMAIL=user@example.org
+ EMAIL=user@example.org
echo "EMAIL = $EMAIL"
+ echo 'EMAIL = user@example.org'
EMAIL = user@example.org

So far so good. The shell isn't mangling anything and we are echoing what we expect.

echo "Email address: here" | perl -p -e "s/here/$EMAIL/"
+ echo 'Email address: here'
+ perl -p -e s/here/user@example.org/
Email address: user.org

Okay, that time the replacement wasn't quoted, so the @example part of the string got expanded (to nothing) and effectively disappeared. Okay, fine, let's escape it with our good friends \Q and \E:

echo "Email address: here" | perl -p -e "s/here/\\Q$EMAIL\\E/"
+ echo 'Email address: here'
+ perl -p -e 's/here/\Quser@example.org\E/'
Email address: user\.org

Well, that was unexpected! \Q and \E quoted the ., but they left the @example part unescaped! What is going on here?

echo "Email address: here" | perl -p -e "s/here/${EMAIL/@/\\@}/"
+ echo 'Email address: here'
+ perl -p -e 's/here/user\@example.org/'
Email address: user@example.org

Okay, so this finally worked, but only because we used bash pattern expansion to do a search and replace. It worked in this particular case because this was an e-mail address. This would be incredibly tedious to do for all possible replacement metacharacters in a more general case.

So again, how do you escape the REPLACEMENT part of a perl subsitution completely when using the s// operator? Is it possible? There has to be a trick I'm missing. =)

SOLVED

ysth's answer suggested using s''', which solves this simple example, but I couldn't use that in my real code, because I needed backreferences in my real use caes. However, ysth's answer and TLP's comment both proposed using $ENV{...}. As far as I can tell, this works perfectly so far in my real use cases, which must be able to use back-references.

Here is an updated version of the example shown above.

$ cat example-new.sh
#!/bin/bash
set -v -x
EMAIL=user@example.org
# Don't touch my delimiters!
echo "Email address goes >>>>>>here<<" | perl -p -e 's/(>+)here(<+)/$1$ENV{EMAIL}$2/'

It works as expected when running it:

$ ./example-new.sh
EMAIL=user@example.org
+ EMAIL=user@example.org
# Don't touch my delimiters!
echo "Email address goes >>>>>>here<<" | perl -p -e 's/(>+)here(<+)/$1$ENV{EMAIL}$2/'
+ echo 'Email address goes >>>>>>here<<'
+ perl -p -e 's/(>+)here(<+)/$1$ENV{EMAIL}$2/'
Email address goes >>>>>>user@example.org<<
wjl
  • 7,519
  • 2
  • 32
  • 41
  • 1
    What on earth are you trying to do here: `${EMAIL/@/\\@}`? – TLP Nov 25 '13 at 00:55
  • 1
    @TLP that is bash syntax for variable expansion with replacement. – wjl Nov 25 '13 at 00:58
  • 1
    Perhaps you should try replace `s/here/$EMAIL/` with `s/here/\$ENV{EMAIL}/`. Which is how you access environment variables in Perl. And/or also change `'` to `"` so you perhaps can remove the backslash. – TLP Nov 25 '13 at 01:04
  • @TLP your comment and ysth's answer both suggested `$ENV{...}` which was a great solution to this problem! – wjl Nov 25 '13 at 01:35
  • 1
    Indeed. You can use lookaround assertions in place of using string captures: `s/(?<=<)here(?=<)/$ENV{EMAIL}/` which saves the trouble of using `$1` etc. – TLP Nov 25 '13 at 01:57

1 Answers1

6

\Q\E are applied to the results of variable interpolation, so you can't keep @example from interpolating that way.

But you can use single quotes:

#!/bin/bash
set -v -x
EMAIL=user@example.org
echo "Email address: here" | perl -p -e "s'here'$EMAIL'"

or, if the email address may contain ' or \\, let perl get $EMAIL from the environment:

export EMAIL=user@example.org
echo "Email address: here" | perl -p -e 's/here/$ENV{EMAIL}/'
ysth
  • 96,171
  • 6
  • 121
  • 214
  • The single-quotes way doesn't work in some real cases because the replacement also uses `$1`, etc, but the `$ENV{...}` method looks like it would work perfectly. – wjl Nov 25 '13 at 01:18
  • that wouldn't work with `$1`; can you show an example of that usage with your expected output? – ysth Nov 25 '13 at 01:22
  • I updated my question with a SOLUTION section to show how your `$ENV{...}` suggestion works with my need for `$1`. – wjl Nov 25 '13 at 01:34
  • ah, I thought you meant $1 in $EMAIL – ysth Nov 25 '13 at 02:47