2

I am working with a Fedora server that runs a customized software package. The server software is quite old, and its database consists of 1,723 files. The database files are constantly changing - they continually grow and changes are not necessarily appended to the end. So right now, we currently back up every 24 hours at midnight when all users are off of the system and the database is in an internally consistent state.

The problem is that we have the potential to lose an entire day's worth of work, which would be unrecoverable. So I'd like to know if there is a way to take some sort of an instantaneous snapshot of these database files that we could back up every 30 minutes or so.

I've read about Linux LVM snapshots, and am thinking that I might be able to do accomplish the goal by taking a snapshot, rsync'ing the files to a backup server, then dropping the snapshot. But I've never done this before,so I don't know if this is the "right" fix.

Any ideas on this? Any better solutions?

Thanks!

Neal
  • 23
  • 2

2 Answers2

6

It all depends on the database. If you don't have a way to tell the database engine "get all your files in a consistent state", then you can't do a snapshot, because the snapshot has no guarantee that the files within it are consistent between themselves -- you could take the snapshot at a moment when one file in the database has been changed, and another one hasn't.

Really, it sounds like you've got a real clusterfudge of a proprietary app, and you'll need to discuss the backup possibilities with the vendor of said app. We won't be able to help you at all without knowing what the nightmare is, and even then...

However, if you can work out a way to tell the database engine "sync all your changes to disk", then yes, an LVM snapshot is a perfectly reasonable way to complete the rest of the data copy. For an example of how to do the whole process (sync to disk, snapshot, mount snapshot, copy off data), take a look at mylvmbackup. It does exactly this process, for MySQL servers.

womble
  • 96,255
  • 29
  • 175
  • 230
  • Yeah, this is not the optimal situation, but I'm trying to avoid a real clusterfudge here. Assuming that the database could be temporarily synced to get it internally consistent, the LVM snapshotting process would allow the server to continue working with the database while we rsync the snapshot, right? I'm just trying to understand if we would need to lock out users during the backup or not. Thanks – Neal Jul 04 '12 at 14:12
  • @Neal: Correct -- once the snapshot is taken, DB operations can continue as normal on the original filesystem, while the contents of the LVM snapshot are mounted elsewhere and copied off to your backup storage location. – womble Jul 04 '12 at 15:06
  • I spoke with the original vendor, and there is no way for them to "sync all changes to disk" in their system. Is there a way to do this through Linux? I'm wondering if there is a system-level way to "lock" all non-open files, then lock all the rest of the files as they are released, and once all of them are locked, snapshot it and do the backup. Is this even possible or am I stretching too far here? – Neal Jul 05 '12 at 01:39
  • 1
    You're stretching *WAY* too far, I'm afraid. If the database starts/stops fairly quickly, perhaps you'll need to gracefully stop the database, do the snapshot, then start it back up again... but that, of course, won't work if the DB process takes more than a second or so to stop/start, or if having it out-of-service for a few seconds will cause Bad Things to happen. Out of curiosity, what's the vendors recommended strategy for doing backups, if they can't ensure that all changes are synced to disk? – womble Jul 05 '12 at 01:50
2

i think the LVM is a good option, but first of all you need to be sure that your files are in a consistent state. So whatever solutions you use if the files are corrupted there is not point having them.

Let's say that you take an snapshot, but at the same time your database is writing some data in the files. If the snapshot is taken when the files are just partially synced you are in trouble.

You know what they say a bad backup is worse than no backup at all.

So I would recommend to find out how can you temporary lock the writing into the database, then sync the disk ,take a snapshot and unlock the database again. The snapshot won't take long so if the users can survive with some delay I would choose this path.

golja
  • 1,621
  • 10
  • 14