2

I have two storage servers at home - one is my workstation, which has a pair of 3TB disks in RAID1.

At first, I stored all of my data there. Recently, I received a small SBC with four 1TB disks. I set them up with RAID0, with the idea being that the SBC remains on at all times for everyone to use (and is faster), and when the workstation is up they sync the differences and the data stays safe there.

Right now, I have some unique data on each. What I'm looking for is a way to do one large transfer of the unique data, and then have something more incremental.

Both servers are formatted with BtrFS.

I had a few options in mind:

  • tar the directories I'm transferring, compress them (with zstd) and then scp or rsync them.
  • send the tar to standard output and pipe it into the compressor and over the network
  • use BtrFS send-receive over the network, either with or without compression
  • use rsync, with or without compression
  • mount one of the server's filesystem on the other with SMB or NFS and do the transfer.

Which of these options do you think would perform best?

Thanks!

dkd6
  • 155
  • 1
  • 9
  • 1
    *Try* what works best for you. Better yet, worry less about performance. Having to restore from other backups (you *do have* other backups, right?) is more expensive, so chose a **robust** method (both robust in that you will not run a bad command by mistake, and robust in that it can handle interruptions gracefully). – anx Sep 26 '20 at 15:34
  • That is good advice - but in my case, everything important is already on the workstation - the other server is just for convenience (I can access it from anywhere, always on etc). I've been using each of these methods at some point for other purposes - so I was just wondering how to maximize the transfer over our (slower) network. – dkd6 Sep 26 '20 at 17:22
  • Another option that you may want to consider is [unison](https://www.cis.upenn.edu/~bcpierce/unison/). Depending on your use case, you may find it very useful - I have. – rickhg12hs Sep 27 '20 at 10:08
  • Looks interesting. I'll give it a spin. Thanks! – dkd6 Sep 27 '20 at 12:18

1 Answers1

0

Raid-0 always scares me as there is no fail-over. I supposed it's ok in this situation since you will have a backup and you can use the other server in an emergency.

My suggestion: Resilio Sync (formerly bit-torrent sync) It is extremely fast and will keep the two up-to-date. There's a free version for linux home use.

https://www.resilio.com/individuals/

Other than that for the destination server, I would suggest

rsync -avz root@sourceserver.com:/path/to/data/ /path/to/data/
Marc Pope
  • 146
  • 8