4

Is it possible to create differential backups based on LVM snapshots (or maybe a similar technique)?

I'd like to perform backups of my intranet server, which hosts multiple services, and their databases (currently using Postgres 8, Postgres 9 and MySQL). So I thought about setting up log shipping based backups (in addition to backing up the file system), but the whole thing is getting a little bit complex.

As an alternative, I'm considering LVM snapshots - but the databases are not small, and I'll need to save backup space...

It's also important, that LVM snapshots are highly safe to use while a database is constantly writing files.

Chris Lercher
  • 4,152
  • 9
  • 35
  • 41

2 Answers2

1

LVM snapshots are copy-on-write at the block device level. Without any special activity in the filesystem or application they behave the same as a crash-and-reboot. If you can quiesce the application and filesystem and flush any pending writes before taking the snapshot you can greatly increase the consistency of the data in the snapshot. For example take a global write lock in the database, then flush the filesystem and make it filesystem read-only (some filesystems like XFS have an explicit quiesce option), make the snapshot then unlock everything.

As far as backup space, snapshots are copy-on-write so you only need enough space allocated to the snapshot to cover the number of blocks which are overwritten on the live filesystem during the lifetime of the snapshot.

mtinberg
  • 1,833
  • 11
  • 9
  • Ok, first of all: Thanks for your answer. I'm not sure, if I fully understand.. 1) "_increase_ the consistency of the data in the snapshot" - I'd like to have a _consistent_ snapshot (which will work, if I restore the data - even if I lose the last few minutes of database commits). 2) Will I be able to restore previous versions, if I overwrite the snapshot? 3) Will it really only copy the differences (if yes, I haven't been able to find the corresponding page in the LVM docs)? – Chris Lercher Jun 29 '11 at 21:20
  • The snapshot will be self-consistant regardless, it will show a point-in-time of what is on the disks. Unfortunately that is the same behavior as if you pulled power and crashed the host, the filesystem journal would need to be replayed and in-flight transactions in the DB rolled back. Quiescing the system by flushing the DB and locking against writes as well as flushing the filesystem gives a higher level of consistency than relying on crash recovery. – mtinberg Jul 05 '11 at 21:04
  • You can have multiple snapshots against the same original block device. Each one needs to be allocated its own snapshot volume space to hold the original data as it gets overwritten and you can keep them around for as long as you want, until they run out of space. Of course a snapshot volume the same size as the original data volume can never run out of space but usually you don't need that much space. As blocks are overwritten in the logical volume they are copied to the snapshot volume so that the snapshot can reconstitute the state of the LV by combining the original and unchanged blocks – mtinberg Jul 05 '11 at 21:10
1

The answer (for me) is simply to create a temporary LVM snapshot, use any incremental backup technique on it, and then discard the snapshot.

As for the safety of LVM snapshots (which I usually have to take while the DB is running), I'm pretty confident that they will work fine. To cover the residual risk, I'm shutting down the database once per week before taking a full backup.

Chris Lercher
  • 4,152
  • 9
  • 35
  • 41