Logrotate isn't deleting old files

Question

I have a daily backup running using logrotate. I don't have root access to this machine to edit the normal logrotate.conf, therefore I'm calling it via crontab like so:

0 4 * * * /usr/sbin/logrotate /home/myuser/backups/logrotate.conf --state /home/myuser/backups/logrotate.tmp

My logrotate.conf looks like this:

/home/myuser/backups/mysql/production/db.sql.gz {
    daily
    nocompress
    dateext
    missingok
    rotate 30
    extension *.sql.gz
    sharedscripts
    postrotate
        mysqldump --login-path=production mydatabase --single-transaction | $(which gzip) > /home/myuser/backups/mysql/production/db.sql.gz
    endscript
}

The issue is that old "logs" (SQL backups) aren't being rotated off/deleted. They just stick around. Running ls -lt in my backup directory gives me some odd results:

-rw-r--r-- 1 myuser mygroup 3799057 Apr 17 04:00 db.sql.gz
-rw-r--r-- 1 myuser mygroup 3799221 Apr 16 04:00 db.sql.gz-20190417
-rw-r--r-- 1 myuser mygroup 3799167 Apr 15 04:00 db.sql.gz-20190416
-rw-r--r-- 1 myuser mygroup 3799090 Apr 14 04:00 db.sql.gz-20190415
-rw-r--r-- 1 myuser mygroup 3799136 Apr 13 04:00 db.sql.gz-20190414
-rw-r--r-- 1 myuser mygroup 3799107 Apr 12 04:00 db.sql.gz-20190413
-rw-r--r-- 1 myuser mygroup 3792122 Apr 11 04:00 db.sql.gz-20190412
-rw-r--r-- 1 myuser mygroup 3794498 Apr 10 04:00 db.sql.gz-20190411
-rw-r--r-- 1 myuser mygroup 3798082 Apr  9 04:00 db.sql.gz-20190410
-rw-r--r-- 1 myuser mygroup 3793882 Apr  8 04:00 db-20190409.sql.gz
-rw-r--r-- 1 myuser mygroup 3793695 Apr  7 04:00 db-20190408.sql.gz
-rw-r--r-- 1 myuser mygroup 3794418 Apr  6 04:00 db-20190407.sql.gz
-rw-r--r-- 1 myuser mygroup 3790567 Apr  5 04:00 db-20190406.sql.gz
-rw-r--r-- 1 myuser mygroup 3792162 Apr  4 04:00 db-20190405.sql.gz
-rw-r--r-- 1 myuser mygroup 3783503 Apr  3 04:00 db-20190404.sql.gz
-rw-r--r-- 1 myuser mygroup 3781192 Apr  2 04:00 db-20190403.sql.gz
-rw-r--r-- 1 myuser mygroup 3782857 Apr  1 04:00 db-20190402.sql.gz
-rw-r--r-- 1 myuser mygroup 3785592 Mar 31 04:00 db-20190401.sql.gz
-rw-r--r-- 1 myuser mygroup 3771475 Mar 30 04:00 db-20190331.sql.gz
-rw-r--r-- 1 myuser mygroup 3771263 Mar 29 04:00 db-20190330.sql.gz
-rw-r--r-- 1 myuser mygroup 3751782 Mar 28 04:00 db-20190329.sql.gz
-rw-r--r-- 1 myuser mygroup 3632604 Mar 27 04:00 db-20190328.sql.gz
-rw-r--r-- 1 myuser mygroup 3744550 Mar 26 04:00 db-20190327.sql.gz
-rw-r--r-- 1 myuser mygroup 3608617 Mar 25 05:00 db-20190326.sql.gz
-rw-r--r-- 1 myuser mygroup 3642558 Mar 24 05:00 db-20190325.sql.gz
-rw-r--r-- 1 myuser mygroup 3600705 Mar 23 05:00 db-20190324.sql.gz
-rw-r--r-- 1 myuser mygroup 3735972 Mar 22 05:00 db-20190323.sql.gz
-rw-r--r-- 1 myuser mygroup 3600285 Mar 21 05:00 db-20190322.sql.gz
-rw-r--r-- 1 myuser mygroup 3737448 Mar 20 05:00 db-20190321.sql.gz
-rw-r--r-- 1 myuser mygroup 3741454 Mar 19 05:00 db-20190320.sql.gz
-rw-r--r-- 1 myuser mygroup 3603502 Mar 18 05:00 db-20190319.sql.gz
-rw-r--r-- 1 myuser mygroup 3617021 Mar 17 05:00 db-20190318.sql.gz
-rw-r--r-- 1 myuser mygroup 3612493 Mar 16 05:00 db-20190317.sql.gz
-rw-r--r-- 1 myuser mygroup 3753894 Mar 15 05:00 db-20190316.sql.gz
-rw-r--r-- 1 myuser mygroup 3742415 Mar 14 05:00 db-20190315.sql.gz
-rw-r--r-- 1 myuser mygroup 3733984 Mar 13 05:00 db-20190314.sql.gz
-rw-r--r-- 1 myuser mygroup 3594213 Mar 12 05:00 db-20190313.sql.gz
-rw-r--r-- 1 myuser mygroup 3594515 Mar 11 05:00 db-20190312.sql.gz
-rw-r--r-- 1 myuser mygroup 3622672 Mar 10 05:00 db-20190311.sql.gz

Am I somehow mis-understanding the behavior of logrotate?

Because `*.sql.gz` is not the extension, `.sql.gz` is. `*.sql.gz` is a glob pattern matching those files, but that's not what is required when telling logrotate what the extension is. — wurtel, Apr 23 '19 at 11:03
I changed the extension, and the weird naming seems to have gone away...however - old logs are still no longer being removed. I currently have 43 in the backups directory. — robertmain, Apr 23 '19 at 23:00

score 2 · Accepted Answer · answered Apr 17 '19 at 16:05

2

Looks like 30 of your files (the oldest ones) are using a naming scheme (db-DATE.sql.gz) and the newer ones are named differently (db.sql.gz-DATE). Also looks like the newer ones have the dateext option applied. This make me think about two possibilities:

You changed the naming scheme before hitting the threshold of files for rotating/deleting, so logrotate will remove the files matching the new naming scheme (db.sql.gz-DATE), but only when there are more than 30 of them (currently there are 9 files). The files with the old naming must be removed manually by you.
You have 2 different logrotate conf files processing the same set of log files. (probably inside /etc/logrotate.d). Maybe when you tested, you ran logrotate /etc/logrotate.d/your_conf.conf and thus bypassed some settings of /etc/logrotate.conf (like dateext). Now crontab is executing the logrotate main file and generated the difference in the files naming.

answered Apr 17 '19 at 16:05

JucaPirama

265
1
7

Yeah, I did notice the weird naming. The only logrotate config I've edited is the one posted above (this is shared hosting so I don't have root access to edit `/etc/logrotate.d/*`). I haven't changed the naming scheme since I wrote the logrotate config. The only thing I've changed at all is the time that the cron runs. Is it possible that the hosting provider is somehow running another logrotate that's found my config file and is conflicting with it? – robertmain Apr 17 '19 at 19:17
I just changed `*.sql.gz` to `.sql.gz` for the extension setting and the file naming has gone back to the old format. Old logs are not being removed though – robertmain Apr 23 '19 at 23:01
1

Just an update to this (over 2 years later!) I left things be for a little while and it does look like it's working after the name change... – robertmain Jul 26 '21 at 19:16

Logrotate isn't deleting old files

1 Answers1