5

Simple regex question:

I want to replace page numbers in a string with pagenumber + some number (say, 10). I figured I could capture a matched page number with a backreference, do an operation on it and use it as the replacement argument in re.sub.

This works (just passing the value):

def add_pages(x):
    return x

re.sub("(?<=Page )(\d{2})",add_pages(r"\1") ,'here is Page 11 and here is Page 78\nthen there is Page 65',re.MULTILINE)

Yielding, of course, 'here is Page 11 and here is Page 78\nthen there is Page 65'

Now, if I change the add_pages function to modify the passed backreference, I get an error.

def add_pages(x):
        return int(x)+10


re.sub("(?<=Page )(\d{2})",add_pages(r"\1") ,'here is Page 11 and here is Page 78\nthen there is Page 65',re.MULTILINE)

ValueError: invalid literal for int() with base 10: '\\1'

, as what is passed to the add_pages function seems to be the literal backreference, not what it references.

Absent extracting all matched numbers to a list and then processing and adding back, how would I do this?

ako
  • 3,569
  • 4
  • 27
  • 38

2 Answers2

7

The actual problem is, you are supposed to pass a function to the second parameter of re.sub, instead you are calling a function and passing the return value.

Why does it work in the first case?

Whenever a match is found, the second parameter will be looked at. If it is a string, then it will be used as the replacement, if it is a function, then the function will be called with the match object. In your case, add_pages(r"\1"), is simply returning r"\1" itself. So, the re.sub translates to this

print re.sub("(?<=Page )(\d{2})", r"\1", ...)

So, it actually replaces the original matched string with the same. That is why it works.

Why it doesn't work in the second case?

But, in the second case, when you do

add_pages(r"\1")

you are trying to convert r"\1" to an integer, which is not possible. That is why it is failing.

How to fix this?

The actual way to write this would be,

def add_pages(matchObject):
    return str(int(matchObject.group()) + 10)
print re.sub("(?<=Page )(\d{2})", add_pages, ...)

Read more about the group function, here

thefourtheye
  • 233,700
  • 52
  • 457
  • 497
  • Great explanation. As an add-on, could I expand to search not just for 2-digit page numbers and still keep the assertion here? – ako Nov 26 '14 at 07:45
  • I didn't get you. Can you please elaborate? – thefourtheye Nov 26 '14 at 07:47
  • Right now I match two-digit numbers with \d{2}. In the actual, non-toy example, I may have 3-digit numbers, too. Variable length patterns cause me trouble. – ako Nov 26 '14 at 07:59
  • @ako You can change it to `\d{2,3}`, so that it will match either two or three digits – thefourtheye Nov 26 '14 at 08:03
  • indeed, tried that one earlier but failed to match digits in the range. I had left the multiline flag on which seemed to be an issue. Turned it off and the repeater worked like a charm. – ako Nov 26 '14 at 21:01
1
def add_pages(matchobj):
    return str(int(matchobj.group(0))+10)


print re.sub("(?<=Page )(\d{2})",add_pages ,'here is Page 11 and here is Page 78\nthen there is Page 65',re.MULTILINE
vks
  • 67,027
  • 10
  • 91
  • 124