4

I've been tasked with setting up a backup system for my small office (around 12 people). Most of our production stuff is on the AWS cloud, so what I need to back up are some small office/development files (under 100G right now), plus our operational VMs and development, which round out to a bit under 1T.

I just need something reliable, convenient, and straightforward. I'm comfortable with Linux, FreeBSD, and to some extent Solaris 10, so I'm leaning toward a full server rather than an appliance system ala Openfiler or FreeNAS.

What I'm contemplating is a small fileserver for general storage and nightly backups of the virtual machines, followed up by an offsite backup to Amazon's S3 storage service. It'd be the usual incremental backups nightly and full backup weekly.

My question is if using ZFS snapshots, both locally and dumped to S3 via 'zfs send [-i]', is a viable backup tool? Or should I stick to using Duplicity, or some other method entirely?

ZFS snapshots on the internal fileserver/backup machine sound like a perfect way to provide quick and convenient data recovery, so I'm likely to go with that for local redundancy. (If you folks see scenarios where relying on ZFS snapshots would be worse than a more traditional archiving backup, feel free to convince me.) But are snapshots flexible enough to lean on for recovery from the loss of my backup server? Or am I better off with something more traditional? (feel free to recommend free or commercial backup solutions you favor.)

AllanA
  • 91
  • 1
  • 4

2 Answers2

2

Keep in mind that unless you quiesce the VMs before you snapshot the ZFS filesystem that their virtual hard disks reside on, you will in effect be making a backup of a crashed system. Also, in order to restore a single file that resides on a ZFS snapshot, you will need to load the entire snapshot somewhere, assuming the worst case of having to pull a snapshot that has been retrieved from remote storage.

While ZFS and snapshots are great for the "uh oh, something really bad happened, and I need the filesystem to be like it was an hour, day week ago", they really aren't a backup solution in the traditional sense. I've used ZFS and snapshots in conjunction with other backup software, such as bacula (www.bacula.org) with good results. It is very handy to be able to zfs send a snapshot of a filesystem off to the backup server where it is then spooled to tape without impacting the production system I/O.

ttyS0
  • 477
  • 2
  • 4
1

I think this is possible.

Consider the halfway point between OpenFiler/FreeNas and full-blown Solaris. The free or commercial NexentaStor is a good appliance solution, or you can use bare-bones (free) Nexenta Core if you prefer to use the server for more than storage. If you go the appliance route, you will be able to schedule your snapshots on whatever interval you'd like (every x minutes, hours, days, etc.) and the replication is fairly clean. You can leverage rsync (or rsync + ssh) or zfs send/receive + netcat or ssh to send to local storage or remote storage. If you use the rsync based replication, your target does not have to be a ZFS filesystem. So that makes it an easy choice for your application.

The other advantage to any recent ZFS solution is that you can use compression and/or deduplication if your spec your storage system well enough in terms of RAM and CPU. This works wonders for certain types of virtual machines and is pretty transparent to users.

If I were planning your backup solution, I'd be sure to have local snapshots on an hourly rollup to daily/weekly/monthly snaps as well as a provision to send the daily snaps to a second local system or a remote system. I'd still augment this with a agent-based backup within the specific virtual machines (e.g. BackupExec dumping to a dedicated backup server + tape drive or rsync of the most important data to another location). With backups, you want options and flexibility for recovery.

Also see the following:

VMware NAS/iSCSI recommendations - smallish organization

Horror stories with Sun ZFS?

http://www.anandtech.com/show/3963/zfs-building-testing-and-benchmarking

Community
  • 1
ewwhite
  • 197,159
  • 92
  • 443
  • 809