0

I am at an impass with my knowledge about bash scripting and rsync (over SSH).

In my use case there is a local folder with log files in it. Those logfiles are rotated every 24 hours and receive a date-stamp in their filename (eg. logfile.DATE) while the current one is called logfile only.

I'd like to copy those files to another (remote) server and then compress those copied log files on this remote server. I'd like to use rsync to ensure if the script does not work once or twice that there are no files skipped (so I would like not to mess with dates and date abbriviations if not nessecary).

However, if I understand correctly, all files would be rsynced, because the already rsynced files do not "match" the rsync algorithm because they are compressed.... How can I avoid that the same file is being copied again, when this very file is on the remote location (only alraedy compressed).

Does someone have an idea or a direction I should focus my research on this? Thank you very much best regards

scheuri
  • 93
  • 1
  • 1
  • 5

1 Answers1

0

When you do the rotation, you rename logfile to logfile.DATE. As part of that operation, use ssh mv to do the same on the archive server at the same time (you can even tell the server to compress it then).

Then you only ever need to rsync the current logfile.

For example, your rotate operation goes from this:

mv logfile logfile.$(date +%F)

To this:

mv logfile logfile.$(date +%F)
ssh archiver mv logfile logfile.$(date +%F) && gzip logfile.$(date +%F)

And your rsync job goes from this:

rsync logdir/ archiver:

To this:

rsync logdir/logfile archiver:
John Zwinck
  • 239,568
  • 38
  • 324
  • 436