How to find/replace and increment a matched number with sed/awk?

Question

Straight to the point, I'm wondering how to use grep/find/sed/awk to match a certain string (that ends with a number) and increment that number by 1. The closest I've come is to concatenate a 1 to the end (which works well enough) because the main point is to simply change the value. Here's what I'm currently doing:

find . -type f | xargs sed -i 's/\(\?cache_version\=[0-9]\+\)/\11/g'

Since I couldn't figure out how to increment the number, I captured the whole thing and just appended a "1". Before, I had something like this:

find . -type f | xargs sed -i 's/\?cache_version\=\([0-9]\+\)/?cache_version=\11/g'

So at least I understand how to capture what I need.

Instead of explaining what this is for, I'll just explain what I want it to do. It should find text in any file, recursively, based on the current directory (isn't important, it could be any directory, so I'd configure that later), that matches "?cache_version=" with a number. It will then increment that number and replace it in the file.

Currently the stuff I have above works, it's just that I can't increment that found number at the end. It would be nicer to be able to increment instead of appending a "1" so that the future values wouldn't be "11", "111", "1111", "11111", and so on.

I've gone through dozens of articles/explanations, and often enough, the suggestion is to use awk, but I cannot for the life of me mix them. The closest I came to using awk, which doesn't actually replace anything, is:

grep -Pro '(?<=\?cache_version=)[0-9]+' . | awk -F: '{ print "match is", $2+1 }'

I'm wondering if there's some way to pipe a sed at the end and pass the original file name so that sed can have the file name and incremented number (from the awk), or whatever it needs that xargs has.

Technically, this number has no importance; this replacement is mainly to make sure there is a new number there, 100% for sure different than the last. So as I was writing this question, I realized I might as well use the system time - seconds since epoch (the technique often used by AJAX to eliminate caching for subsequent "identical" requests). I ended up with this, and it seems perfect:

CXREPLACETIME=`date +%s`; find . -type f | xargs sed -i "s/\(\?cache_version\=\)[0-9]\+/\1$CXREPLACETIME/g"

(I store the value first so all files get the same value, in case it spans multiple seconds for whatever reason)

But I would still love to know the original question, on incrementing a matched number. I'm guessing an easy solution would be to make it a bash script, but still, I thought there would be an easier way than looping through every file recursively and checking its contents for a match then replacing, since it's simply incrementing a matched number...not much else logic. I just don't want to write to any other files or something like that - it should do it in place, like sed does with the "i" option.

Answer 1

I think finding file isn't the difficult part for you. I therefore just go to the point, to do the +1 calculation. If you have gnu sed, it could be done in this way:

sed -r 's/(.*)(\?cache_version=)([0-9]+)(.*)/echo "\1\2$((\3+1))\4"/ge' file

let's take an example:

kent$  cat test 
ello
barbaz?cache_version=3fooooo
bye

kent$  sed -r 's/(.*)(\?cache_version=)([0-9]+)(.*)/echo "\1\2$((\3+1))\4"/ge' test     
ello                                                                             
barbaz?cache_version=4fooooo
bye

you could add -i option if you like.

edit

/e allows you to pass matched part to external command, and do substitution with the execution result. Gnu sed only.

see this example: external command/tool echo, bc are used

kent$  echo "result:3*3"|sed -r 's/(result:)(.*)/echo \1$(echo "\2"\|bc)/ge'       

gives output:

result:9

you could use other powerful external command, like cut, sed (again), awk...

Answer 2

Pure sed version:

This version has no dependencies on other commands or environment variables. It uses explicit carrying. For carry I use the @ symbol, but another name can be used if you like. Use something that is not present in your input file. First it finds SEARCHSTRING<number> and appends a @ to it. It repeats incrementing digits that have a pending carry (that is, have a carry symbol after it: [0-9]@) If 9 was incremented, this increment yields a carry itself, and the process will repeat until there are no more pending carries. Finally, carries that were yielded but not added to a digit yet are replaced by 1.

sed "s/SEARCHSTRING[0-9]*[0-9]/&@/g;:a {s/0@/1/g;s/1@/2/g;s/2@/3/g;s/3@/4/g;s/4@/5/g;s/5@/6/g;s/6@/7/g;s/7@/8/g;s/8@/9/g;s/9@/@0/g;t a};s/@/1/g" numbers.txt

Answer 3

This perl command will search all files in current directory (without traverse it, you will need File::Find module or similar for that more complex task) and will increment the number of a line that matches cache_version=. It uses the /e flag of the regular expression that evaluates the replacement part.

perl -i.bak -lpe 'BEGIN { sub inc { my ($num) = @_; ++$num } } s/(cache_version=)(\d+)/$1 . (inc($2))/eg' *

I tested it with file in current directory with following data:

hello
cache_version=3
bye

It backups original file (ls -1):

file
file.bak

And file now with:

hello
cache_version=4
bye

I hope it can be useful for what you are looking for.

---

UPDATE to use File::Find for traversing directories. It accepts * as argument but will discard them with those found with File::Find. The directory to begin the search is the current of execution of the script. It is hardcoded in the line find( \&wanted, "." ).

perl -MFile::Find -i.bak -lpe '

    BEGIN { 
        sub inc { 
            my ($num) = @_; 
            ++$num 
        }

        sub wanted {
            if ( -f && ! -l ) {  
                push @ARGV, $File::Find::name;
            }
        }

        @ARGV = ();
        find( \&wanted, "." );
    }

    s/(cache_version=)(\d+)/$1 . (inc($2))/eg

' *

Answer 4

This is ugly (I'm a little rusty), but here's a start using sed:

orig="something1" ;
text=`echo $orig | sed "s/\([^0-9]*\)\([0-9]*\)/\1/"` ;
num=`echo $orig | sed "s/\([^0-9]*\)\([0-9]*\)/\2/"` ;
echo $text$(($num + 1))

With an original filename ($orig) of "something1", sed splits off the text and numeric portions into $text and $num, then these are combined in the final section with an incremented number, resulting in something2.

Just a start since it doesn't consider cases with numbers within the file name or names with no number at the end, but hopefully helps with your original goal of using sed.

This can actually be simplified within sed by using buffers, I believe (sed can operate recursively), but I'm really rusty with that aspect of it.

Answer 5

perl -pi -e 's/(\?cache_version=)(\d+)/$1.($2+1)/ge' FILE [FILE...]

or for a complete solution:

find . -type f | xargs perl -pi -e 's/(\?cache_version=)(\d+)/$1.($2+1)/ge'

perl substitution operator

• /e modifier evaluates the replacement as if it were a Perl statement, using its return value as the replacement text.
• . operator concatenates strings in Perl. The parentheses ensures that the arithmetic operation $2+1 takes precedence over concatenation.
• /g modifier applies substitution to all matched strings within line

perl options

• -p ensures that perl will execute the command on every line of each file
• -i ensures that each file will be edited inplace
• -e specifies the perl command(s) that are executed (in this case, the substitution operation)