Logrotate creates long names and does not delete logs

Question

postgres_db_dump_20091016.gz.1.1.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021
postgres_db_dump_20091017.gz.1.1.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021
postgres_db_dump_20091018.gz.1.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021
postgres_db_dump_20091019.gz.1.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021
postgres_db_dump_20091020.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021
postgres_db_dump_20091021.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021

This is sample output (filenames) when I run logrotate by force. Somehow it's not recognising the old files.

Debug output sample:

glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding old rotated logs failed
renaming /var/backup/postgres/postgres_db_dump_20091019.gz.1.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021 to /var/backup/postgres/postgres_db_dump_20091019.gz.1.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021
rotating log /var/backup/postgres/postgres_db_dump_20091020.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021, log->rotateCount is 4
dateext suffix '-20091021'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding old rotated logs failed
renaming /var/backup/postgres/postgres_db_dump_20091020.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021 to /var/backup/postgres/postgres_db_dump_20091020.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021
rotating log /var/backup/postgres/postgres_db_dump_20091021.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021, log->rotateCount is 4
dateext suffix '-20091021'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding old rotated logs failed
renaming /var/backup/postgres/postgres_db_dump_20091021.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021 to /var/backup/postgres/postgres_db_dump_20091021.gz.1.1.1.1.1.1.1.1.1-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021-20091021

And configuration file:

daily
rotate 4
create 666 root root
maxage 30
dateext
compress
missingok
notifempty
noolddir

# system-specific logs may be also be configured here.
/var/log/* {
    size=10M
    postrotate 
        /usr/bin/killall -HUP syslog-ng
    endscript
}

/var/backup/postgres/postgres* {
    nocompress
    nocreate
}

score 8 · Answer 1 · answered Oct 20 '09 at 23:20

8

/var/log/* {
    size=10M
    postrotate 
        /usr/bin/killall -HUP syslog-ng
    endscript
}

The /var/log/* is your problem. That's matching all the files in /var/log, including the already-compressed files. You'll need to refine that glob matching pattern to only match the non-compressed files.

answered Oct 20 '09 at 23:20

EEAA

109,363
18
175
245

No, because the ones that are growing are in /var/backup, not in /var/log. – Paul Tomblin Oct 20 '09 at 23:23
2

Okay, fair enough. The same holds true for his /var/backup/postgres/postgres* directive as well, though. – EEAA Oct 20 '09 at 23:30
...to expand on that, if there are any log files in /var/log, he should be seeing those being re-compressed and renamed in the same manner. – EEAA Oct 20 '09 at 23:31
I'm surprised that it's not happening in /var/log as well. Perhaps the files are just not getting large enough? There is a 10M limit there. – Jeff Snider Oct 20 '09 at 23:42
1

this is a correct answer, you should consider it accepting. your wildcard matches previously-rotated files too. – asdmin Oct 21 '09 at 07:31

score 8 · Answer 2 · answered Oct 20 '09 at 23:22

8

The manpage for logrotate says

   Please  use wildcards with caution.  If you specify *, logrotate will rotate all files,
   including previously rotated ones.  A way around this is to use the olddir directive or
   a more exact wildcard (such as *.log).

answered Oct 20 '09 at 23:22

Paul Tomblin

5,225
1
28
39

2

i always like when someone pinpoints something from man which solves write-only users' question ;) – asdmin Oct 21 '09 at 07:32

Jeff Snider · Accepted Answer · 2009-10-21T01:22:42.440

The other answers here are correct, you need to change what files are being matched by your config. The same problem could occur with the files in /var/log, but the combination of compression and the 10M size limit is keeping it from happening.

There's another problem you're going to run into that I'd like to point out. Logrotate is built to rely on a consistent log file name to rotate out old files. When it processes a log file, it uses that as a base name to find all old versions of it by searching for the specific extension that old versions would have. That's what the glob stuff in your debug log is looking for. Since your filename changes every time because of the date, it will never go back and look at those old ones because the filename it's working on now doesn't match.

There's two things you could do. First option would be to set up postgres or whatever script is creating that dump file to not use the date. Keeping the filename consistent will let logrotate do it's thing and clear out the old ones. Alternatively, you could skip logrotate entirely and put something like this in your crontab:

0 4 * * * find /var/backup/postgres/postgres_db_dump_*.gz -mtime +10 -delete

That would run every night and delete the dump files that are older than 10 days.

Thank you for well explained answers, indeed the wildcard was causing rotated files to rerotate. Jeff pointed out issue with postgres dump, no date should be included in filename in order to be consistent. Lesson learned, I hope someone else find this information useful. — iElectric, Oct 21 '09 at 16:23

Logrotate creates long names and does not delete logs

3 Answers3