1

How "expensive" is an rsync on the various computing resources (CPU, RAM, network, etc.) if nothing has changed on the source file system?

I've two weak servers across a very high latency network and files show up sporadically (due to a process out of my control), but I really need to sync aggressively.

I'm doing a basic recursive send on an entire folder.

How does this change if there are 100s of GBs or maybe TBs of data sitting (again, mostly unchanged) on the source filesystem?

The source may be a CIFS mounted share, which does not work with inotify, which is why I'm asking about rsync performance characteristics, not alternative mirroring solutions.

Anthony Mastrean
  • 441
  • 1
  • 8
  • 18
  • Maybe see http://serverfault.com/a/688660/984 (ie Lsyncd). – Zoredache Dec 16 '16 at 17:42
  • 1
    Are you using `--checksum`? If so, then the impact is also to CPU for the md4sum's. If not, it's the disk IO for the metadata of the files. On SSD's, not so much of an issue. On spinning rust, it can cause a bit of thrashing and over time, extra wear on the drives if you do this frequently. – Aaron Dec 16 '16 at 18:07
  • Wouldn't `inotify` be better for this scenario? – dawud Dec 16 '16 at 18:13
  • 2
    While it's relatively light on the network, it does hit the storage pretty good just to do the comparison. Not something you want to do every 15 minutes unless you've got _lots_ of IOPS to spare (and you don't). And if the storage is network mounted, then it won't be relatively light there either... You should really be looking for another solution. – Michael Hampton Dec 16 '16 at 21:05

0 Answers0