6

New to Python so please forgive my ignorance. I'm trying to modify backreferenced strings in a regular expression.

Example:

>>>a_string
'fsa fad fdsa dsafasdf u.s.a. U.S.A. u.s.a fdas adfs.f fdsa f.afda'
>>> re.sub(r'(?<=\s)(([a-zA-Z]\.)+[a-zA-Z]\.{0,1})(?=\s)', '<acronym>'+re.sub(r'\.',r'',(r'\1').upper())+'</acronym>', a_string)
'fsa fad fdsa dsafasdf <acronym>u.s.a.</acronym> <acronym>U.S.A.</acronym> <acronym>u.s.a</acronym> fdas adfs.f fdsa f.afda'

Instead of the output I desire:

'fsa fad fdsa dsafasdf <acronym>USA</acronym> <acronym>USA</acronym> <acronym>USA</acronym> fdas adfs.f fdsa f.afda'

Thanks for your help.

dreftymac
  • 31,404
  • 26
  • 119
  • 182
MattL
  • 61
  • 1

4 Answers4

3

From the docs:

If repl is a function, it is called for every non-overlapping occurrence of pattern. The function takes a single match object argument, and returns the replacement string. For example:

And see the example contained in the linked docs.

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
  • Forgive my ignorance, but I don't know how this helps me. The example doesn't use a backreference. When I take the backreference out and, say insert a dummy string, it works fine. The problem is in the modification of the backreferenced string (I think). – MattL Sep 02 '11 at 04:42
  • Did you try returning a replacement string? – Ignacio Vazquez-Abrams Sep 02 '11 at 05:12
3

As Ignacio Vazquez-Abrams suggested, you can solve your problems by passing a callable function to re.sub(). I figured that sample code would explain it best, so here you go:

import re

s = "fsa fad fdsa dsafasdf u.s.a. U.S.A. u.s.a fdas adfs.f fdsa f.afda"

s_pat = r'(?<=\s)(([a-zA-Z]\.)+[a-zA-Z]\.{0,1})(?=\s)'
pat = re.compile(s_pat)

def add_acronym_tag(match_object):
    s = match_object.group(0)
    s = s.replace('.', '').upper()
    return "<acronym>%s</acronym>" % s

s = re.sub(pat, add_acronym_tag, s)
print s

The above prints:

fsa fad fdsa dsafasdf <acronym>USA</acronym> <acronym>USA</acronym> <acronym>USA</acronym> fdas adfs.f fdsa f.afda

So you aren't actually modifying the backreference, because strings are immutable. But this is just as good: you can write a function to do any processing you want, and then return whatever you want, and that is what re.sub() will insert in the final result.

Note that you can use regular expressions inside your function; I just used the .replace() string method because you just want to get rid of a single character, and you don't really need the full power of regular expressions for that.

steveha
  • 74,789
  • 21
  • 92
  • 117
  • Thanks, that helps a lot. But I'm still not sure why my initial attempt didn't work. Don't re.sub() and upper() count as callable functions? – MattL Sep 02 '11 at 14:58
  • They count as callable functions. But `re.sub()` calls the function and passes a single argument: a "match object". `re.sub()` and `str.upper()` don't know what to do with a match object. Look again at my function; the first thing it does is extract the string from the match object. So, you need to write a wrapper function that calls `re.sub()` or `str.replace()` and `str.upper()`; the wrapper needs to take a match object as an argument; and the wrapper needs to not take any other arguments to tell it what to do. – steveha Sep 02 '11 at 18:33
2

The "modifying a backreference" needs re-phrasing as you seem to confuse the notions.

A replacement backreference is a special combination of characters iniside a string that tells the regex engine to refer to some specific capturing group values (aka submatches) retrieved during a match operation.

When you use r'\1'.upper(), you are trying to make the \1 string uppercase, and as \1 has no uppercasable letters, you get \1 as a result, and this \1 - unchanged - is applied as the (part of) string replacement pattern.

That is why you can't modify the capturing group value this way.

That is why you have to use a callable as the replacement argument (see Ignacio's answer): you need to pass the match object to the re.sub to be able to manupilate the submatches (although you may of course replace a char or two in a backrefence, say, r'\g<12>'.replace('2','1') to "obfuscate" \g<11> backreference, but there is little sense in this operation).

Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563
1

Context

  • python 3.x
  • using re.sub to do regular expression substitution
  • apply arbitrary modification on part of a string, without modifying the whole string

Problem scenario

  • UserMattL/uu002matt1675257544 wants to match part of a string with regex
  • user wants to modify the matched part of the string

Solution

  • The general solution to this scenario is already posted elsewhere in this thread
  • This answer shows a simpler example that does basically the same thing, but using python lambda instead of declaring a standalone function

Example

  • user has a string matching a MSFT Windows filepath specification
  • user wants to change the drive letter to lowercase, but without modifying any other part of the string
  • regex is appropriate here, since the drive letter can be any character, making str.replace() impractical
  import re
  ss7676demotest = 'D:/AlphaOne/BravoTwo'
  rx7676demotest = re.compile(r'^(\w):')
  ss7676demotest = re.sub(rx7676demotest, lambda obmatch: '{vjj}:'.format(vjj=obmatch.group(1).lower()), ss7676demotest,)
  print(ss7676demotest) ## d:/AlphaOne/BravoTwo

Rationale

  • python lambda allows user to minimize the amount of code

Pitfalls

  • the question talks about modifying a backreference, but that's actually not what's going on here
    • backreference is a regex concept for specifying subregions of a string that match a specific part of a regex
    • in this context, we want to specify one of those matched subregions so we can modify it without affecting any other part of the string
    • we do that with python regex.match() object (eg, obmatch.group(1) in this example)
  • python lambda is practical in simple scenarios that do not involve a lot of logic, but not always favorable
    • for more elaborate transformations, it is favorable to write a standalone function instead of using a lambda
    • to enhance readability
    • to make the code more maintainable
    • see the other answer in this thread for the alternate non-lambda approach
dreftymac
  • 31,404
  • 26
  • 119
  • 182