44

I am using rsync in a bash script to keep files in sync between a few servers and a NAS. One issue I have run into is trying to generate a list of the files that have changed from the during the rsync.

The idea is that when I run rsync, I can output the files that have changed into a text file - more hoping for an array in memory - then before the script exists I can run a chown on only the changed files.

Has anyone found a way to perform such a task?

# specify the source directory
source_directory=/Users/jason/Desktop/source

# specify the destination directory
# DO NOT ADD THE SAME DIRECTORY NAME AS RSYNC WILL CREATE IT FOR YOU
destination_directory=/Users/jason/Desktop/destination

# run the rsync command
rsync -avz $source_directory $destination_directory

# grab the changed items and save to an array or temp file?

# loop through and chown each changed file
for changed_item in "${changed_items[@]}"
do
        # chown the file owner and notify the user
        chown -R user:usergroup; echo '!! changed the user and group for:' $changed_item
done
Chris Stryczynski
  • 1,566
  • 2
  • 20
  • 25
Jason M.
  • 565
  • 1
  • 5
  • 7
  • → see [my anwer here](http://serverfault.com/a/834981/88431). Also `-i` for itemize, but with a few more twists... – Frank N Feb 26 '17 at 12:45

4 Answers4

67

You can use rsync's --itemize-changes (-i) option to generate a parsable output that looks like this:

~ $ rsync src/ dest/ -ai
.d..t.... ./
>f+++++++ newfile
>f..t.... oldfile

~ $ echo 'new stuff' > src/newfile

~ $ !rsync
rsync src/ dest/ -ai
>f.st.... newfile

The > character in the first position indicates a file was updated, the remaining characters indicate why, for example here s and t indicate that the file size and timestamp changed.

A quick and dirty way to get the file list might be:

rsync -ai src/ dest/ | egrep '^>'

Obviously more advanced parsing could produce cleaner output :-)

I came across this great link while trying to find out when --itemize-changes was introduced, very useful:

http://andreafrancia.it/2010/03/understanding-the-output-of-rsync-itemize-changes.html (archived link)

knarf
  • 141
  • 7
Kyle Smith
  • 9,683
  • 1
  • 31
  • 32
  • 3
    For a cleaner output as mentioned, `rsync -zaic src/ dest/ | grep '^?c' | cut -d' ' -f2 --dry-run` would only list the modified files (**different checksum**) , definitely a keeper, thx :) FYI putting `--dry-run` after the command instead of using the `n` option is for me a best practice – MediaVince Dec 23 '16 at 09:37
  • Essentially the same as `rsync -zavc src/ dest/ --dry-run` without verbose – MediaVince Dec 23 '16 at 09:45
  • 2
    Awesome, I didn't know the `!` syntax for using previous parameters... Thanks a lot! – Niloct Oct 13 '20 at 02:42
20

Use the -n flag, combined with the -c checksum flag and the -i flag:

# rsync -naic test/ test-clone/
>fcst...... a.txt
>fcst...... abcde.txt
>fcst...... b.txt

In this example, three files have changed, based on the contents (as determined by the checksum) of the file itself. However, no file syncing is done because of the -n flag

Bonus

If you want to run chown on the changed files, parse them out with sed or similar and process with xargs, for example:

rsync -naic test/ test-clone/ | sed 's/............//' | xargs -I+ sudo chown root "test-clone/+"
Seamus
  • 266
  • 1
  • 6
JDS
  • 2,598
  • 4
  • 30
  • 49
9

This question is a little bit old, but I think it worth to be added:

-i is a shortcut of --out-format=%i%n%L

And %n means the filename, (section log format of man rsyncd.conf)

P.S. rsync version 3.1.0

Cychih
  • 191
  • 1
  • 3
2

Summarizing a few other answers (especially @Cychih's), you can get the list of changed files like so:

rsync --out-format='%n' src/ dest/

Which will print only the changed files, eg;

rsync --out-format='%n' src/ dest/
a.txt
bcde.txt
b.txt

You can save that to a list this way:

changed_items=($(rsync --out-format='%n' src/ dest/))
for item in "${items[@]}"; do
   echo $item
   echo $item
done

You can pipe them to another command like so:

rsync --out-format='%n' src/ dest/ | xargs open

Note that it's very common to include -acz (archive, checksum, and compress) flags as well.

rattray
  • 121
  • 2