Logrotate: Rotating non-log files?

Question

I have a backup script that compresses various files and directories and creates .tgz archives. Files are named, e.g.

...
backup_2010-10-28.tar.gz
backup_2010-10-29.tar.gz
backup_2010-10-30.tar.gz
backup_2010-10-31.tar.gz
backup_2010-11-01.tar.gz

I want to manage these files so only the last 5 backups are kept, with older files being deleted.

Can I use logrotate to do this? They aren't log files and are already compressed. They're in /root and not in /var/log - can I still use it?

Thanks

mark · Answer 1 · 2010-11-01T11:45:19.870

Without a change to your process, logrotate on its own will not do what you're looking for here. The key problem here is that, while logrotate can take wildcards, it will not treat the files as one if you do so and will instead attempt to rotate all of them individually, which is definitely NOT what you want.

You can, however, make it work the way you describe as long as the most recent backup is created without a date stamp. If you backup process creates /root/backup.tar.gz for instance, you could use the following logrotate configuration:

/root/backup.tar.gz {
    rotate 5
    nocompress
    dateext
    dateformat _%Y-%m-%d
    extension .tar.gz
    missingok
}

The quick rundown of the options here:

rotate 5 -- keep 5 rotations before deleting
nocompress -- do not compress the files after rotating
dateext -- use the date as the rotation extension instead of incrementing numbers
dateformat _%Y-%m-%d -- set the date extension format you want to use
extension .tar.gz -- make the .tar.gz come after the rotation extension
missingok -- if the file we want to rotate isn't there, don't worry about it and move on (the default is to throw an error)

Hope this helps!

Nice work. I learned a couple of things by reading this. I do want to add though, that some distributions, notably RedHat EL, tend to strip out options that aren't "Enterprise Ready," so YMMV. — zerolagtime, Nov 01 '10 at 18:09
According to this solution http://stackoverflow.com/questions/14858752/logrotate-to-clean-up-date-stamped-files logrotate could accomplish the task. — Pieter, Oct 24 '16 at 10:33

score 26 · Answer 2 · answered Nov 01 '10 at 10:24

26

You don't have to use logrotate to do it. Just use a command like this:

ls -1 /root/backup_* | sort -r | tail -n +6 | xargs rm > /dev/null 2>&1

This command will leave the most recent 5 files and remove the remaining (if any). You can use it in a cron job.

answered Nov 01 '10 at 10:24

Khaled

36,533
8
72
99

but this doesn't rotate.... – simpleuser Mar 18 '17 at 21:03
Doesn't answer the question. – kubanczyk May 22 '17 at 02:39
5

The question was to keep five most recent backups an this does exactly that. Even simpler: ls -t1 /root/backup_* | tail -n +6 | xargs rm -f – yoyoma2 Apr 16 '18 at 15:36
This answered *my* question. Sadly the original question was the closest I could find with Google-fu. – Sukima Jun 20 '19 at 17:03

score 3 · Accepted Answer · answered Nov 01 '10 at 10:18

Logrotate rotates files, so the answer is yes - probably, and if no sufficient permissions then place them in /backup or something. Check what group and user the rotated logs have :-).

There's options for compression in logrotate, så if "compress" is NOT configured - well then it won't try. Also in your case, "rotate 5" option.

Take a look in /etc/logrotate.d (or where ever it's stored in your system)

score 0 · Answer 4 · answered Mar 08 '18 at 23:15

I just had the same situation. Logrotate sounds pretty cool but it didn't work for me at 100% because it mismatch data and file names.

So, to avoid confusion, I decided to incorporate the following line to execute after created my current backup in order to keep the last 5 backups.

My logs are i.e.:

tbl-bcx-20180308_010501.tar.bz2
tbl-bcx-20180307_010501.tar.bz2
tbl-bcx-20180306_010501.tar.bz2

New line into my backup-script (based on a comment above)

find /BCX/dumpsql/ -type f | sort -r | tail -n +6 | xargs rm

Regards,

score -1 · Answer 5 · edited May 22 '17 at 13:55

You can rotate it manually.

rotating_logger()
{

_LOG_FILE_=${1}
_MESSAGE_TO_ECHO_=${2}

_LOG_FILE_BYTES_SIZE_=`wc -c ${_LOG_FILE_} | awk '{print $1}'`

#echo "_LOG_FILE_BYTES_SIZE_ : ${_LOG_FILE_BYTES_SIZE_}"

# in MB
if [ ${_LOG_FILE_BYTES_SIZE_} -gt ${_FILE_SIZE_IN_BYTES_} ] ; then
        #cat /dev/null > ${_LOG_FILE_}
         for (( i=${_LOG_ROTATE_MAX_INDEX_}; i>=1; i-- ))
        do
                 if [ $i -eq ${_LOG_ROTATE_MAX_INDEX_} -a -f ${_LOG_FILE_}.$i ] ; then
                        rm -rf ${_LOG_FILE_}.$i
                 fi

                if [ $i -gt 10 ] ; then
                        rm -rf ${_LOG_FILE_}.10
                elif [ $i -eq 9 -a -f ${_LOG_FILE_}.9 ] ; then
                       mv ${_LOG_FILE_}.9 ${_LOG_FILE_}.10
                elif [ $i -eq 8 -a -f ${_LOG_FILE_}.8 ] ; then
                       mv ${_LOG_FILE_}.8 ${_LOG_FILE_}.9
                elif [ $i -eq 7 -a -f ${_LOG_FILE_}.7 ] ; then
                       mv ${_LOG_FILE_}.7 ${_LOG_FILE_}.8
                elif [ $i -eq 6 -a -f ${_LOG_FILE_}.6 ] ; then
                       mv ${_LOG_FILE_}.6 ${_LOG_FILE_}.7
                elif [ $i -eq 5 -a -f ${_LOG_FILE_}.5 ] ; then
                       mv ${_LOG_FILE_}.5 ${_LOG_FILE_}.6
                elif [ $i -eq 4 -a -f ${_LOG_FILE_}.4  ] ; then
                       mv ${_LOG_FILE_}.4 ${_LOG_FILE_}.5
                elif [ $i -eq 3 -a -f ${_LOG_FILE_}.3  ] ; then
                       mv ${_LOG_FILE_}.3 ${_LOG_FILE_}.4
                elif [ $i -eq 2 -a -f ${_LOG_FILE_}.2  ] ; then
                       mv ${_LOG_FILE_}.2 ${_LOG_FILE_}.3
                elif [ $i -eq 1 -a -f ${_LOG_FILE_}.1 ] ; then
                       mv ${_LOG_FILE_}.1 ${_LOG_FILE_}.2
                fi
        done
        mv ${_LOG_FILE_} ${_LOG_FILE_}.1
        touch ${_LOG_FILE_}
fi

echo `date +%Y-%m-%d:%H:%M:%S` : "${_MESSAGE_TO_ECHO_}" >> ${_LOG_FILE_}
#echo "" >> ${_LOG_FILE_}
}

Now to put data to the log, just do

rotating_logger <file_path> "Welcome world!"

Perhaps you could put a little effort into formatting your post, if you want it to be useful. — Tim, May 22 '17 at 00:32

Logrotate: Rotating non-log files?

5 Answers5