Can sed regex simulate lookbehind and lookahead?

Question

I'm trying to write a sed script that will capture all "naked" URL's in a text file and replace them with <a href=[URL]>[URL]</a>. By "naked" I mean a URL that is not wrapped inside an anchor tag.

My initial thought was that I should match URL's that do not have a " or a > in front of them, and also do not have a < or a " after them. However, I am running into difficulty with expressing the concept of "do not have in front of or behind" because as far as I know sed does not have look-ahead or look-behind.

Sample Input:

[Beginning of File]http://foo.bar arbitrary text
http://test.com other text
<a href="http://foobar.com">http://foobar.com</a>
Nearing end of file!!! http://yahoo.com[End of File]

Sample Desired Output:

[Beginning of File]<a href="http://foo.bar">http://foo.bar</a> arbitrary text
<a href="http://test.com">http://test.com</a> other text
<a href="http://foo.bar">http://foo.bar</a>
Nearing end of file!!! <a href="http://yahoo.com">http://yahoo.com</a>[End of File]

Observe that the third line is unmodified, because it is already inside <a href>. On the other hand, both the first and second lines are modified. Finally, observe that all non-URL text is unmodified.

Ultimately, I am trying to do something like:

sed s/[^>"](http:\/\/[^\s]\+)/<a href="\1">\1<\/a>/g 2-7-2013

I began by verifying that the following will correctly match and remove a URL:

sed 's/http:\/\/[^\s]\+//g'

I then tried this, but it is not able to match URL's that start at the beginning of file / input:

sed 's/[^\>"]http:\/\/[^\s]\+//g'

Is there a way to work around this in sed, either by simulating lookbehind / lookahead, or explicitly matching beginning of file and end of file?

I'm looking for a URL that is not preceded by quotation mark or a greater than sign. — merlin2011, Feb 15 '13 at 01:35
Update your question to show some representative sample input and the expected output given that input - that's more important for us to see than what you have tried (though that's useful too). — Ed Morton, Feb 15 '13 at 02:49
@EdMorton, observe that the question has been updated with sample input and output. — merlin2011, Feb 15 '13 at 08:21

Ed Morton · Accepted Answer · 2013-02-15T17:38:13.273

sed is an excellent tool for simple substitutions on a single line, for any other text manipulation problems just use awk.

Check the definition I'm using in the BEGIN section below for a regexp that matches URLs. It works for your sample but I don't know if it captures all possible URL formats. Even if it doesn't though it may be adequate for your needs.

$ cat file
[Beginning of File]http://foo.bar arbitrary text
http://test.com other text
<a href="http://foobar.com">http://foobar.com</a>
Nearing end of file!!! http://yahoo.com[End of File]
$
$ awk -f tst.awk file
[Beginning of File]<a href="http://foo.bar">http://foo.bar</a> arbitrary text
<a href="http://test.com">http://test.com</a> other text
<a href="http://foobar.com">http://foobar.com</a>
Nearing end of file!!! <a href="http://yahoo.com">http://yahoo.com</a>[End of File]
$
$ cat tst.awk
BEGIN{ urlRe="http:[/][/][[:alnum:]._]+" }
{
    head = ""
    tail = $0
    while ( match(tail,urlRe) ) {
       url  = substr(tail,RSTART,RLENGTH)
       href = "href=\"" url "\""

       if (index(tail,href) == (RSTART - 6) ) {
          # this url is inside href="url" so skip processing it and the next url match.
          count = 2
       }

       if (! (count && count--)) {
          url = "<a " href ">" url "</a>"
       }

       head = head substr(tail,1,RSTART-1) url
       tail = substr(tail,RSTART+RLENGTH)
    }

    print head tail
}

In the url regex you use `_` as a valid host name character, shouldn't it be `-`? — Camilo Martin, Feb 05 '15 at 12:24
As I say at the top of the answer `Check the definition I'm using in the BEGIN section below for a regexp that matches URLs. It works for your sample but I don't know if it captures all possible URL formats.`. I'm no URL syntax expert. — Ed Morton, Feb 05 '15 at 13:39

SwiftMango · Answer 2 · 2013-02-15T01:43:20.027

2

The obvious problem with your command is

You did not escape the parenthesis "("

This is the weird thing about sed regex. It is different to Perl regex that many symbols are by default "literal". You have to escape them to "function". Try:

s/\([^>"]\?\)\(http:\/\/[^\s]\+\)/\1<a href="\2">\2<\/a>/g

edited Feb 15 '13 at 01:43

answered Feb 15 '13 at 01:36

SwiftMango

15,092
13
71
136

As a clarification, I am trying to match URL's that do not have a " or a > in front of them. – merlin2011 Feb 15 '13 at 01:40
The given solution will not match `http://google.com` at the beginning of file or beginning of input. – merlin2011 Feb 15 '13 at 01:41
@merlin2011 I see what you mean. sed does not support look ahead/behind, I just edited. The question mark makes it optional – SwiftMango Feb 15 '13 at 01:43
4

About the weird `\(`, an option is to use `sed -r` so that `(` doesn't need to be quoted. (I even have a `rsed` alias) – darque Feb 15 '13 at 05:44
@texasbruce, when you make it optional, it now has the effect that it will match URL's inside ` – merlin2011 Feb 15 '13 at 08:17
Incidentally, you can use the `-E` flag to use "modern" regex. Then you don't need to escape brackets. – abalter Sep 15 '16 at 17:40

Can sed regex simulate lookbehind and lookahead?

2 Answers2