3

What's the safest way to backup a graphite whisper database, is it safe to just use tar or is there another method required to prevent corruption?

Ward - Trying Codidact
  • 12,899
  • 28
  • 46
  • 59
Rwky
  • 774
  • 1
  • 8
  • 17
  • I use `rsync` and do incremental/delta backups rather than `tar`. See my answer here: https://serverfault.com/questions/741346/rsync-directory-so-all-changes-appear-atomically/741420#741420 That said, you'll need _something_ to dump the data to a flat file that understands the database and can put it in a stable state or take it offline (e.g. locks against others writing while it's dumping). I found: https://github.com/jjneely/whisper-backup but I have no idea if this works or not. – Craig Estey May 14 '16 at 05:02

1 Answers1

2

There are a few options for backing up databases like this, and assuming there's an easy mechanism to ensure consistency (temporarily halting the database/locking write access, etc), they're all roughly similar.

The key thing (as Craig Estey mentions in his comment) is to use some mechanism to ensure what you write is consistent. There's not point backing up a database if you're going to get junk!

I did find this link from Swift that shows how they leveraged LD_PRELOAD and flock() to lock the Whisper files and back up the databases in a consistent state. Looks like they also open sourced the code!

In general for backing up stuff like this though:

Tar it up

Pros:

  • Simple
  • Portable
  • No additional tools required
  • Compression rate is flexible (xz, gzip, bz2)

Cons:

  • Potentially slow
  • Incremental backups are possible but a bit of a faff

Rsync it

Pros:

  • Also simple
  • Fast
  • Incremental backups are easy
  • Compression in transit

Cons:

  • Uses SSH so if you're not copying to a remote server might be slower than other tools
  • Compression not available at end without using tar/gzip etc.

Filesystem Snapshots

Pros:

  • Almost instant
  • Simple
  • Probably space efficient?

Cons:

  • Requires setting up the drives with LVM/similar
  • Not as simple as other solutions.
shearn89
  • 3,403
  • 2
  • 15
  • 39