0

We are taking a backup of our tool's home directories and configuration within same S3 bucket separating by directory names. We have versioning enabled on S3 bucket. With the fear of loosing important files in S3 backup as well, if something goes wrong with our distributed file system and scheduled S3 sync job runs - We didn't enable --delete option during s3 sync.

Now, I wonder - If something goes wrong with the local backups and we realise - let's say 4 days after - and we need to restore 4 days old files within just one directory of S3 bucket, how I can achieve that?

Also, if you can answer - Since we didn't enable --delete option, restore will also restore all files which were deleted or not present locally - which are no longer required - How can we avoid that during restore?

EnigmaticNeo
  • 45
  • 1
  • 1
  • 6
  • I'd recommend that you add more sophistication asap to your backup strategy. Using the awscli to sync to S3 is not going to be as good as something like cloudberry lab, duplicati or duplicity. – jarmod Aug 27 '18 at 23:57
  • Thanks for valuable suggestion. Will definitely those out too and try to implement that simultaneously while we improve existing backup strategies. – EnigmaticNeo Aug 28 '18 at 00:25

1 Answers1

1

For business-critical backups, I would recommend using a "real" backup tool that can provide point-in-time restoration.

There are many backup tools that can store data in Amazon S3 and Amazon Glacier, such as CloudBerry Backup.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
  • We have nomad in place where we run backup process inside the container - separated by application backups. This is very dynamic environment with lots of different possibilities to run those different backup containers on different nomad clients. Data we are talking about is in size of at least 20TB. Cloudberry charges (correct me if I'm wrong) for data size it backs up and for number of linux machines where it's installed. Overall, it becomes extremely expensive just to have "Point in Time" restoration facility adoption for particular sub-directory in S3. – EnigmaticNeo Aug 28 '18 at 02:36