0

I am running this script via a daily cronjob to backup a wordpress site to Amazon S3

* 4 * * * /root/s3backyupsync.sh > /dev/null 2>&1

Contents of .sh script

#!/bin/bash
################################################
# Simple script to synchronise data into S3 bucket(s)
#############################################
LOCALDIR='/var/www/html'
S3BUCKET='s3://my-backup/'
MAILLOG=no
MAILRECEIPIENT=''
MAILSUBJECT='Daily S3 Cloud Backup Sync'
#############################################

### Remember start time
CURRENTDATETIME=$(date +%Y-%m-%-d-%H:%M:%S);

### Sync data
s3cmd --exclude "/wp-content/cache/*" sync --no-progress --recursive --skip-existing --no-check-md5 $LOCALDIR $S3BUCKET;

### Mailing
if [ "${MAILLOG}" = "yes" ]; then
        TMPFILE=/tmp/s3backupsynclog.txt;
        echo "STARTED: ${CURRENTDATETIME}" > $TMPFILE;
        echo "ENDED  : $(date +%Y-%m-%-d-%H:%M:%S)" >> $TMPFILE;
        echo '' >> $TMPFILE;
        s3cmd du -H $S3BUCKET >> $TMPFILE;
        echo '' >> $TMPFILE;
        rm $TMPFILE;
fi

This however, doesn't update modified files with their new versions.

Also files and folders which no longer exist at the source, are not deleted in Amazon S3.

In general, my script also consumes a lot of resources, so I am wondering if I need to be changing some options here.

JoaMika
  • 1,727
  • 6
  • 32
  • 61
  • 1
    At a guess, after taking a look at [this page](http://s3tools.org/usage) I would say that `skip-existing` is the issue - it won't overwrite an existing file. Update the script based on the link and you can remove files that are no longer used. – stdunbar Sep 07 '17 at 18:08
  • Yes but with the '--no-check-md5', does that mean that simply all files will be overwritten every time ? This will increase the server load. – JoaMika Sep 07 '17 at 19:59
  • The md5 check would not be used in the case of skip-existing. But I'd encourage you to post on the [s3cmd forums](https://sourceforge.net/p/s3tools/discussion/618865/) as they might have a better solution. Alternately, you could use the normal [AWS CLI](https://aws.amazon.com/cli/) that has a `sync` command build in to it. – stdunbar Sep 07 '17 at 20:30
  • These days, it is recommended to use the official [AWS Command-Line Interface (CLI)](http://aws.amazon.com/cli/) rather than `s3cmd`. – John Rotenstein Sep 08 '17 at 02:07
  • John, I have looked up some turorials about AWS CLI but they all talk about backing up files as a .tar archive. I am looking to mirror my entire wordpress installation to s3. – JoaMika Sep 08 '17 at 12:15

0 Answers0