3

I have a directory that has many folders with code inside it, and I need to run a script to rsync changes over to a couple other servers. I have that script written and working fine, but I cannot find something that will check the directory tree for changes. I tried stat, but that only check one folder down, not recursively (as far as I could figure out). Anyone know of a command or program I could install to do this? I use Ubuntu 10.04. Thanks in advance.

Mike
  • 31
  • 2

6 Answers6

3

inotifywait will do recursive checks.

It's in the Ubuntu repositories.

Dennis Williamson
  • 62,149
  • 16
  • 116
  • 151
3

The perfect tool for this job is using the inotify kernel service. You can use them on a shell script with the package inotify-tools (debian/ubuntu). You can read more and see some examples on the project's page. There's even an example on the page that seems to do something close to what you want:

#!/bin/sh

# get the current path
CURPATH=`pwd`

inotifywait -mr --timefmt '%d/%m/%y %H:%M' --format '%T %w %f' \
-e close_write /tmp/test | while read date time dir file; do

       FILECHANGE=${dir}${file}
       # convert absolute path to relative
       FILECHANGEREL=`echo "$FILECHANGE" | sed 's_'$CURPATH'/__'`

       rsync --progress --relative -vrae 'ssh -p 22'  $FILECHANGEREL usernam@example.com:/backup/root/dir && \
       echo "At ${time} on ${date}, file $FILECHANGE was backed up via rsync"
done
coredump
  • 12,713
  • 2
  • 36
  • 56
  • 1
    The problem here is if the files are actively changing and `rsync` takes significant time to complete, an unrecoverable backlog could be created. See [my answer here](http://serverfault.com/questions/161974/question-about-inotifywait/161992#161992) and the attached comments. Great care should be taken that `rsync` completes quickly or a `while inotifywait ...` loop (without the `-m` switch) should be used. Avoiding the temptation to background the `rsync` should be carefully considered. – Dennis Williamson Jul 20 '10 at 19:13
  • Hear what Dennis is saying, its wise advice. Maybe if he makes a script that accumulates changes and then `rsync` then from time to time to prevent multiple processes... – coredump Jul 20 '10 at 21:14
2

rsync will check the directory tree for changes and copy only the changed files if you give it a directory as the source.

user9517
  • 115,471
  • 20
  • 215
  • 297
  • True, but what if there are new enough files that it takes more than a minute to copy? I also want this to be able to run as soon as changes happen, not a minute later. This copies code to live webservers, and having them out of sync for a minute is not ideal. – Mike Jul 20 '10 at 17:26
  • Before I learned about inotify and inotifywait I'd have said if that happens split the rsync source directories. Inotify is clearly the way forward though. – user9517 Jul 20 '10 at 19:04
1

Have a look at inotify. I solved a similar problem some time ago with dnotify or sgi_fam (don't recall which) but inotify seems to have superceeded them both.

AndyN
  • 1,769
  • 12
  • 14
1

Someone already invented this wheel. It's called lsyncd and it is apparently a mashup of rsync and inotify. I haven't used it myself, because the limitations of inotify are such that it doesn't scale to really large (and deep) directories. I'm waiting until they reimplement it directly using fsnotify.

There's also PIrsyncD which seems to do the same thing.

lukecyca
  • 2,205
  • 13
  • 20
0

use find

find /path -mmin 1

That will check if a file or directory was modified 1 min ago, look at the man page for more options.

topdog
  • 3,520
  • 17
  • 13
  • Is there a way to write this script so that basically if that command returns anything, then run my rsync script, else sleep for a minute? – Mike Jul 20 '10 at 17:11