0

I have a btrfs-filesystem consisting of several harddrives in which is stored about 11 TB of Data. My backup consists of a NAS which exports one path via NFS. The path is then mounted on the machine with the btrfs-bilesystem and rsync is called to keep the nfs export synced to the main filesystem. I call rsync with one -v and send the results of the run to my email account to be sure everything is synchronized correctly. Now by pure chance I found out that some directories were not synchronized correctly - the directories existed on the NAS but they were empty. It is most likely not a rights issue since rsync is run as root. So it seems that in my situation rsync is not entirely trustworthy but I would like to compare the two directory trees to see if there are any files missing on the NAS and/or if there are files which dont exists on the btrfs anymore and which should have been deleted by rsync according. (I use the --delete option).

I am therefore looking for for a program or a script which can help me to check is rsync is running correctly. I don't need anything complicated like checksums, all I want to know if the NAS contains all the files in the btrfs-filesystem.

Any suggestions where to start looking?

Yours, Stefan

Stefan Malte Schumacher
  • 1
  • 1

1 Answers1

0

Run the following commands to list all files:

find /path/to/fs -type f | sort > filesystem.txt
find /path/to/nfs -type f |sort > nfs.txt

Then compare the lists:

diff -u filesystem.txt nfs.txt
Roland Smith
  • 42,427
  • 3
  • 64
  • 94
  • I worked on the problem and it seems that rsync is not recognizing files in one directory as it should. Thank you for the tip with diff by the way, Roland. Running rsync with rsync --delete -avvv /mnt/btrfs-raid/Downloads/ /mnt/qnap/Downloads output lines like recv_files(ATOS/00014.m2ts) recv_files(ATOS/00037.clpi) recv_files(ATOS/00037.m2ts) and finished with: recv_files phase=1 generate_files phase=2 send_files phase=2 send files finished total: matches=0 hash_hits=0 false_alarms=0 data=0 recv_files phase=2 recv_files finished generate_files phase=3 generate_files finished – Stefan Malte Schumacher May 26 '16 at 22:13
  • sent 4,865 bytes received 59,368 bytes 128,466.00 bytes/sec total size is 345,102,247,470 speedup is 5,372,662.77 [sender] _exit_cleanup(code=0, file=main.c, line=1183): about to call exit(0) Still, on the qnap the directory from which the files are exists but is empty. – Stefan Malte Schumacher May 26 '16 at 22:15
  • By default rsync uses a "quick-check" algorithm where it only compares file size and last-modified time to decide if a file has changed. You could try running rsync with the `-c` option so it calculates and compares a checksum to decide which file has changed. That will run slower obviously. – Roland Smith May 26 '16 at 22:36
  • 1
    Found the problem. It was not rsync's fault after all; samba on the qnap did not show the files in the directories, but samba on the fileserver did. I had to change the permissions on the fileserver and after the next run they were applied to the copy on the qnap and did show up on samba on qnap as they should. – Stefan Malte Schumacher May 27 '16 at 17:26
  • Great! It would have been bad had you found a bug in rsync! – Roland Smith May 27 '16 at 19:25