Split large Logfile with Suffix like from logrotate

Question

after I've seen that I forgot to add a logrotate-conf for php5-fpm.log on one Server, and now seeing that it has grown REALLY big, I'm asking myself how to split this Logfile into separate ones, with a Suffix like logrotate would produce. You know...like this mistake never happened :-P

But the closest that I came up with is:

split -C 10m -d php5-fpm.log php5-fpm.log.

...but this is producing files like:

php5-fpm.log.00
php5-fpm.log.01
php5-fpm.log.02
...

cutting off the leading zeros from the suffix would be a simple task, and could easily be done manually. But here is my problem: How could I revert the suffix, so that the file with the highest suffix becoming the one with the lowest??

Jakov Sosic · Accepted Answer · 2015-09-20T15:19:43.760

2

To cut the suffix length (eliminate zeroes), you can do the following:

$ split --suffix-length=1 -C 10m -d php5-fpm.log php5-fpm.log.

Unfortunately there is no param you could pass to split, which would invert the suffixes, but you could use tac instead of cat, and invert the file and pass it to split command, something like:

$ tac php5-fpm.log | split --suffix-length=1 -C 10m -d - php5-fpm.log.

Now all you need to do is invert split files once more (and compress it while we're at it) :)

$ for i in php5-fpm.log.*; do tac $i | gzip > $i.gz; rm -f $i; done

EDIT:

There's another approach someone could take for this one, in case you don't want to use tac:

split --suffix-length=1 -C 10m -d php5-fpm.log php5-fpm.
export COUNT=$(expr $(ls php5-fpm.[0-9] | wc -l) - 1)
for i in $(seq 0 $COUNT); do
    mv php5-fpm.$i php5-fpm.log.$(expr $COUNT - $i);
done

First command will split the big log file into smaller ones: php5-fpm.0 php5-fpm.1 php5-fpm.2 ... later, you count the number of files and only invert numbers with power of bash expression evaluation: expr(1).

edited Sep 20 '15 at 15:19

answered Sep 20 '15 at 15:11

Jakov Sosic

5,267
4
24
35

Thank you very much for your Answer, Jakov!...and especially for the loop including gzip at the end :-) – mr.alex Sep 20 '15 at 15:16
No problem. I added another possible approach if someone has religious problems with tac :) – Jakov Sosic Sep 20 '15 at 15:20
The second approach will destroy half of your files because it replaces file `php5-fpm.log.$(expr $COUNT - $i)` with file `php5-fpm.log.$i` for $i < $COUNT. – Tilman Schmidt Sep 20 '15 at 15:26
@TilmanSchmidt - it won't. Please try reading the second part again. – Jakov Sosic Sep 20 '15 at 15:55
1

Ah, you're right. My fault for not looking properly. – Tilman Schmidt Sep 20 '15 at 15:58

score 0 · Answer 2 · answered Sep 20 '15 at 15:31

You can achieve the renaming part with this quick and dirty shell script:

#!/bin/bash
for x in php5-fpm.log.*
do
  max=${x/php5-fpm.log./}
done
mkdir renamed-files || exit 1
for x in php5-fpm.log.*
do
  num=${x/php5-fpm.log./}
  let num=max-num+1
  mv $x renamed-files/php5-fpm.log.$num
done
mv renamed-files/* .
rmdir renamed-files

Note the renamed files being collected in a subdirectory to avoid name collisions with files not yet renamed.

Note also the exit after the mkdir in case subdirectory renamed-files cannot be created, for example because a file of that name exists. Continuing the script might have irreversible undesired effects in that case.

Split large Logfile with Suffix like from logrotate

2 Answers2