0

I have an EC2 instance with data I want to sync to a mounted, but remote, volume, as a backup.

rsync seems like the way to go with this, so as a test I took my test file (a Postgres pg_dump file) and used rsync -v to copy it to the mounted volume:

[ec2-user work]$ rsync -v dump.sql.1 ../backup/dump.sql
dump.sql.1

sent 821704315 bytes  received 31 bytes  3416650.09 bytes/sec
total size is 821603948  speedup is 1.00

Then, I ran it again, expecting to see minimal sent/received numbers because it would just be checksums. Instead...

[ec2-user work]$ rsync -v dump.sql.1 ../backup/dump.sql
dump.sql.1

sent 821704315 bytes  received 31 bytes  3402502.47 bytes/sec
total size is 821603948  speedup is 1.00

I'm new to rsync so perhaps I'm missing something, but isn't the idea that the source and destination files are checked for differences, and then a patch is generated and applied to the destination? Why is this not reducing the amount of data 'sent' to just the size of the checksums?

Some background if it's relevant:

the mounted volume is using s3fs, mounted with s3fs <bucketname> backup.

sharakan
  • 115
  • 5

1 Answers1

3

rsync by default copies eveerything. You can use flags like -t or -c to make it check timestamps of files and don't re-transfer files who's last modification timestamp hasn't changed (-t), or where the checksum is still the same (-c). Another useful option is -a (I always use it), which sets -t and a few other options useful for archiving/backups.

Dennis Kaarsemaker
  • 19,277
  • 2
  • 44
  • 70
  • Ha, sounds like I should've RTFM more carefully. rsync -ac looks like it does the job for me, thanks! – sharakan Nov 30 '12 at 21:36
  • Careful with -c -- it's expensive and really only needed if files can have the same timestamp, but differing contents. – Dennis Kaarsemaker Nov 30 '12 at 21:41
  • Turns out I was wrong, with -c or without still doesnt' work. But the problem, it turns out, is with s3fs, not resync usage. See rsync related issues here: http://code.google.com/p/s3fs/issues/list?can=2&q=rsync&colspec=ID+Type+Status+Priority+Milestone+Owner+Summary&cells=tiles – sharakan Dec 05 '12 at 20:01