shell: log directory rotation, more compact code avoiding code duplication

Question

in my shell scripts I need to rotate log directories. I am looking for a more compact, scaleable and elegant way than this, but have currently no idea how to solve this i.e. in a while loop and to calculate with variables.

function f_rotate_logdirs()
{
        if [ -d $LOGDIR_OLD14 ]; then
                # be extra cautious, no rm -rf operation ...
                rm -rf $LOGDIR_OLD14
        fi
        if [ -d $LOGDIR_OLD13 ]; then
                mv $LOGDIR_OLD13 $LOGDIR_OLD14
        fi
        [...]
        if [ -d $LOGDIR_OLD1 ]; then
                mv $LOGDIR_OLD1 $LOGDIR_OLD2
        fi
        if [ -d $LOGDIR ]; then
                mv $LOGDIR $LOGDIR_OLD1
        fi
        mkdir -p $LOGDIR
        echo $DATE > $LOGDIR/0.DATE
}

Do you have an idea for a more compact code which easily scales up to n_days ? Any help on this would be much appreachiated. Many thanks for this upfront.

Or the logrotate tool ? https://www.tutorialspoint.com/unix_commands/logrotate.htm — nullPointer, Feb 22 '19 at 10:15
Sorry, but logrotate is no feasible solution, because I need to rotate directories and this needs to happen during execution of the script. So it needs to be done inside of the script. — ennox, Feb 22 '19 at 12:34
I do not know how the modulo operator would help here. Can you please detail your proposal ? At the moment I do not understand what you mean. — ennox, Feb 22 '19 at 12:36
`$LOGDIR_OLD1 $LOGDIR_OLD2` have you heard of arrays? `mv $LOGDIR $LOGDIR_OLD1` this is very bad for applications that hold their logs open all the time. — KamilCuk, Feb 22 '19 at 20:59
To be clear about what KamilCuk is referring to above, a well-written application will keep writing to the same file after it's renamed, unless it's explicitly triggered or signaled to reopen its output files, because reopening an output file every time one wants to write a new line of logging to it is _extremely_ inefficient, and file handles are attached to inodes, not the directory entries that point to them. — Charles Duffy, May 23 '22 at 03:47

KamilCuk · Answer 1 · 2019-02-23T14:54:04.563

0

The following works:

rm -r -f LOGDIR15
seq 1 14 | tac | xargs -t -n1 sh -c 'if [ -d "$1$2" ]; then mv -n "$1$2" "$1$(($2+1))"; fi' -- LOGDIR
if [ -d "LOGDIR" ]; then mv -n "LOGDIR" "LOGDIR1"; fi
mkdir LOGDIR

We need to handle the first and last separately. Making a function out of it, would be:

backup() {
    rm -r -f "$1$2"
    seq 1 "$(($2 - 1))" | tac | 
    xargs -t -n1 sh -c 'if [ -d "$1$2" ]; then mv -n "$1$2" "$1$(($2+1    ))"; fi' -- "$1"
    if [ -d "$1" ]; then mv -n "$1" "$1"1; fi
    mkdir "$1"
}

with usage:

backup LOGDIR 15

would move the directory named LOGDIR to LOGDIR1 and LOGDIR2 and .. LOGDIR15.

It seems unknown to me, why you use variables $LOGDIR_OLD15 and not just directory names themselves.

edited Feb 23 '19 at 14:54

answered Feb 22 '19 at 21:12

KamilCuk

120,984
8
59
111

Many thanks for your kind proposal. BTW I am using variables to have them at the beginning of the script to make the script easier customizeable for other purposes or environments. It's not the only shell function in the script. – ennox Feb 23 '19 at 14:34
But does it make sense? I suspect `LOGDIR=some_dir` and `LOGDIR_OLD1=some_dir_old1` and `LOGDIR_OLD2=some_dir_old2` ie. the pattern is repetitive. Why not just `LOGDIR=some_dir` `OLDSUFFIX=_old` and then use `${LOGDIR}${OLDSUFFIX}2`. I would also advise to keep a convention , where upper case variables are globals, and lower case variables are locals to the function. – KamilCuk Feb 23 '19 at 14:51
Yes correct makes sense, the many LOGDIR definitions are from the 1st version where this all was kind of static and no while or for loop. – ennox Feb 24 '19 at 15:46

ennox · Answer 2 · 2019-02-26T14:57:10.737

Thanks for your input, the commands seq and tac were new to me. Now I found some time to code it newly and I took some of your nice ideas. My aim was to get a scaleable solution which is easy to understand and maintain.

I decided to name the current logdir "$dir.0" because some of my scripts gather config diffs of devices that are being taking on daily basis. This makes coding a little bit easier for getting the diffs between i.e. "5 and 3 days ago" or between "2 days ago and today".

#! /bin/sh

# Customizeable settings
LOGDIR_BACKUPS=14                       # max number of backups

# Directories
LOGDIR_NAME=logs
LOGDIR=$LOGDIR_NAME.0

# Defines
DATE=`date +'%Y%m%d-%H%M'`

function f_rotate_logdirs() {
        local dir=$1                    # name of logdir folder
        local max=$2                    # max #
        local min=0                     # $dir.0 = current logdir

        for i in `seq $min $max | tac`
        do
                case $i in
                        $max)   if [ -d $dir.$i ]; then
                                        rm -f $dir.$i/*
                                        rmdir $dir.$i
                                fi
                                ;;
                        $min)   if [ -d $dir.$i ]; then
                                        mv $dir.$i $dir.$((i+1))
                                fi
                                mkdir $dir.$i
                                echo $DATE > $dir.$i/0.DATE
                                ;;
                        *)      if [ -d $dir.$i ]; then
                                        mv $dir.$i $dir.$((i+1))
                                fi
                                ;;
                esac
        done
}

f_rotate_logdirs $LOGDIR_NAME $LOGDIR_BACKUPS

shell: log directory rotation, more compact code avoiding code duplication

2 Answers2