0

I have an EC2 server running Sendy (bulk mailing) and I wanted to take backups of the SQL db. Currently, I can download the backup.sql file manually via Virtualmin but I wanted to automate this.

I've been looking at Glacier since it's low cost and I won't need to regularly retrieve the backup; it's only for emergencies. I've looked online a lot and I'm getting mixed ideas on how to do this. Some say that you can only backup from S3 to Glacier. In my case that means EC2 -> S3 -> Glacier but I don't really need the S3 and find it uneccesary to pay for it.

Is it possible to automate sending the backup.sql file from EC2 to Glacier without S3?

Or if you think there is a better way of handling this, then I'm open to ideas.

nbz
  • 103
  • 2

1 Answers1

0

There are two AWS services here that are relavent: - Amazon Glacier, recently called Amazon S3 Glacier, is made for long term corporate archiving. You shouldn't use it unless you fully understand the service, the vaults, the archives, etc. - Amazon S3 is a much more flexible service, file / object based. It has a number of storage classes, including standard, infrequent access (IA), Glacier, and deep archive. This is the service you should use.

Using AWS S3 you can simply upload your backup file. You can use different file names based on the date, or you can overwrite the old file and have versioning turned on.

You could use an incremental backup tool such as Restic, which will reduce your storage requirements, and can age out the backups (eg grandfather, father, son scheme). If you use Restic you have to be sure not to put files such as index into glacier class due to the recall time - best keep them in IA class or similar. You can technically move the data files to a glacier type class, but the minimum storage time can negate the savings. Simply storing the files into S3 deep archive class may be simpler and cheaper.

Either way, this is simple to achieve. You have a cron job that does the database export then uses the S3 API to upload the file to S3. Here's how I do it for mysql with restic

mysqldump --skip-dump-date -h localhost db_name > /var/backups/database/database-name.sql
restic -q --repo s3:s3.amazonaws.com/bucketname/foldername backup /var/backups/database --exclude="*.tmp" --exclude="thumbnails" --cleanup-cache

Alternately if you want to do a basic S3 upload, the command line I use on Windows is this - Linux is similar

aws s3 sync c:\backupfolder s3://bucket-name/ --profile AWS-cli-profile-name --storage-class DEEP_ARCHIVE --exclude *.txt

In both cases I turn on bucket versioning, but it's especially critical if you just upload the files without an incremental backup like Restic. In that case you will probably want to delete old versions otherwise costs can increase significantly over time.

Tim
  • 31,888
  • 7
  • 52
  • 78
  • Thanks for the quick response. The only file I have is backup.sql which right now is fairly small since I just set up the server. Hence I'm trying to make "my mistakes" now. So with this .sql file, does your comment for the index apply? Sorry my knowledge in this area isn't very strong. – nbz Feb 05 '20 at 08:34
  • Also, I was following this getting started guide - https://docs.aws.amazon.com/amazonglacier/latest/dev/introduction.html It says S3 Glacier but it doesn't talk about buckets etc. Is that different from you saying S3 with storage class Glacier? – nbz Feb 05 '20 at 08:36
  • The comment about the index was specifically referring to the Restic index files, which are used if you do incremental backups to S3, not a database index. If you do incremental backups using Restic or similar you don't use Glacier class, you use S3 standard or IA class. I'll edit my comment about service names. – Tim Feb 05 '20 at 08:48
  • Thanks for updating the answer. It makes a lot more sense now. I'm going to assume that the S3 bucket will still be charged at Glacier pricing if my storage class is Glacier? – nbz Feb 05 '20 at 08:54
  • S3 pricing is [here](https://aws.amazon.com/s3/pricing/). Deep Archive is cheaper than the standalone glacier service. If you're happy with the answer please accept it :) – Tim Feb 05 '20 at 17:24