0

Say I am running an HTTP server with data at /var/www. I want to backup /var/www to /root/backup/.tmp/var/www (and then tar them to somewhere) daily automatically.

Mostly the backup is using rsync technique. The problem is that since the HTTP server is running, there could be file modification during an rsync backup process.

For an HTTP server a certain "transaction" could involve multiple files, e.g. modifying file A and B at once, and therefore such scenario is possible: rsync backups file A => a transaction occurs and file A and B are modified => rsync backups file B. This causes the backup-ed files to be inconsistent (A is before transaction while B is after transaction).

For an HTTP server shutting down for backup is mostly not viable. Is there a way to avoid such inconsistent file backup?

This also applies to potentially other services like FTP (a certain "transaction" could be an upload of a folder which contains multiple files) or so.

Danny Lin
  • 111
  • 3
  • That's why many people flush write caches to disk before making a snapshot (with a volume manager or from their storage array) and run consistent file level backups using that snapshot. – HBruijn Aug 09 '16 at 11:04

1 Answers1

0

It depends on your environment, but basically it is not possible with rsync only. If your filesystem supports snapshots, use it for freeze whole filesystem state in one point in time and backup from that snapshot. If it is not possible, you have to look if inconsistency is really problem (it isn't in 90% of environments I've seen) and if yes, you will need to shut down new connections for this server during backups. It should be no problem, because your customers will be served from another server in your pool (if you have only one server, you have bigger issue that inconsistent backups).

Ondra Sniper Flidr
  • 2,653
  • 12
  • 18
  • A filesystem snapshot takes zero time so that it's impossible that a file be modified during the snapshot process? – Danny Lin Aug 09 '16 at 16:00
  • My case is one server only, so shutting down the service for backup is not viable. Currently my strategy is to backup regularly and let my PC download the backup archives automatically when it's online. – Danny Lin Aug 09 '16 at 16:03
  • Yep, filesystem snapshot is atomic. It works just as freezing "inodes" and new modifications are saved into separated. You have to make backups with filesystem tools but it is safe. See btrfs or zfs for more info. – Ondra Sniper Flidr Aug 09 '16 at 16:42