33

I've noticed several posts on this site which say that with gnu sed you should use ( and ) in regex rather than \( and \). But then I looked in the gnu sed manual and saw that they specify that \( and \) must be used. What's up?

jww
  • 97,681
  • 90
  • 411
  • 885
grok12
  • 3,526
  • 6
  • 26
  • 27
  • You need to work on your backslashes. – chrisdowney Jun 17 '11 at 15:16
  • Something happened and I cannot tell the difference between your use of ( and ). As for me I always escape parens when using regexp's. – mrK Jun 17 '11 at 15:16
  • Can you link to some of the posts you mention, for example? – razlebe Jun 17 '11 at 15:16
  • I've always escaped parens in any regexp. If the manual says to do it, do it. – Rafe Kettler Jun 17 '11 at 15:17
  • @chrisdowney I suppose that I should have used double back slashes, is that right? Apparently murgatroid99 fixed them because they look ok now. – grok12 Jun 17 '11 at 15:24
  • 1
    I put ` around your parentheses so they would be treated as code – murgatroid99 Jun 17 '11 at 15:27
  • @mrK @razlebe @Rafe: I should have pointed out that on my system (mac OSX, bash 3.2) back slashing the parens does not work. – grok12 Jun 17 '11 at 15:27
  • In most regular expressions in other languages (e.g. JS, C#, php) the unescapsed '(' and ')' are used for capturing, whereas the backslashed versions '\(' and '\)' are used to capture the parenthesis characters. For some odd reason in sed this is reversed (unless you use extended regular expressions). – sprite Sep 20 '16 at 09:04

4 Answers4

33

This part of the gnu sed manual you linked to explains that whether you should escape parentheses depends on whether you are using basic regular expressions or extended regular expressions. This part says that the -r flag determines what mode you are in.

Edit: as stated in grok12's comment, the -E flag in bsd sed does what the -r flag does in gnu sed.

murgatroid99
  • 19,007
  • 10
  • 60
  • 95
  • 1
    Thank you, murgatroid. I now realize that your answer is correct for gnu sed but not for bsd sed which uses the -E flag to do what -r does in gnu sed. – grok12 Jun 17 '11 at 18:29
22

Originally sed, like grep and everything else, used \( to indicate grouping, whereas ( just matched a literal open-paren.

Many newer implementations of regular expressions, including egrep and perl, switched this around, so \( meant a literal open-paren, and ( was used to specify grouping.

So now with GNU sed, ( is a special character; just like egrep. But on other systems (e.g. BSD) it's still the old way, as far as I can tell. Unfortunately this is a real mess, because now it's hard to know which one to use.

MattDMo
  • 100,794
  • 21
  • 241
  • 231
chrisdowney
  • 1,528
  • 1
  • 11
  • 21
  • Thanks, chris. I now realize that your answer is correct except that for BSD sed there is a -E option to enable the modern (extended) regex. – grok12 Jun 17 '11 at 18:32
  • 2
    About "So now with gnu sed, ( is a special character; just like egrep" - this doesn't seem to be the case, at least on my version gnu sed (Ubuntu 12.04, sed version 4.2.1). The default is still "basic regexep" beaning `()` are interpreted literally. I need `-r` to enable extended mode. – BeeOnRope May 19 '15 at 08:24
11

Thanks to rocker, murga, and chris. Each of you helped me understand the issue. I'm answering my own question here in order to (hopefully) put the whole story together in one place.

There are two major versions of sed in use: gnu and bsd. Both of them require parens in basic regex to be escaped when used for grouping but not escaped when used in extended regex. They diff in that the -r option enables extended regex for gnu but -E does so for bsd.

The standard sed in mac OSX is bsd. I believe much of the rest of the world uses gnu sed as the standard but I don't know precisely who uses what. If you are unsure which you are using try:

> sed -r

If you get a 

> sed: illegal option -- r

reply then you have bsd.

grok12
  • 3,526
  • 6
  • 26
  • 27
6

Escaped parentheses (\() make the regex search for parentheses as part of the expression.

Unescaped parentheses (() make the regex group the contents of the parentheses together.

In other words, if you escape them, the engine looks for them, but if you leave them as is, they cause the engine to group results into variables.

An example to demonstrate:

$myString = "junk(150)moar";

To get just the number:
#^\w+\((\d+)\)\w+$#

($1 is 150)

It's a mess, I know, but it demonstrates the use of grouping parentheses and parentheses as part of the matching expression.

Update Years Later:

As user @bmk correctly points out, this answer applies to extended regular expressions, but not to basic regular expressions. It's difficult to find basic regular expressions as the default parsing engine in most programming languages, etc., but it would be prudent to verify which engine you are using before assuming this answer will apply to your situation.

Community
  • 1
  • 1
rockerest
  • 10,412
  • 3
  • 37
  • 67
  • After learning more I now realize that you answer is correct for basic regex but not for extended regex. – grok12 Jun 17 '11 at 18:26
  • 6
    @grok12: Actually the answer is true for extended regexps but not for basic ones... – bmk Jun 17 '11 at 19:02
  • Is the opposite true for basic regular expressions? i.e. use \\( for grouping and ( for matching? – Samuel Sep 29 '15 at 22:39
  • @Samuel I don't actually know the ins and outs of regex engines, but my hypothesis is that basic regular expressions very likely *do not perform grouping*. Basic regular expression engines are generally a "one-way" algorithm. In other words, once a character has been passed, it is forgotten about. This would preclude grouping from working, since the engine needs to collect what it has passed over in case it needs to scoop it up into a back-reference. In other words: no, the opposite is not the case, and in fact there is no inverse in basic engines, because they operate differently. – rockerest Oct 09 '15 at 19:04