0

I have a simple script to pull data from a remote server as a process generates it using rsync:

while :
do
    rsync -avz --remove-source-files -e ssh me@remote:path/to/foo* ./

    rsync -avz --remove-source-files -e ssh me@remote:path/to/bar* ./

    rsync -avz --remove-source-files -e ssh me@remote:path/to/baz* ./

    rsync -avz --remove-source-files -e ssh me@remote:path/to/qux* ./

    sleep 900 #wait 15 minutes, try again
done

If there are no files, rsync returns exit status 12 (apparently). In the event that none of the above calls to rsync finds any data, I would like to break from the loop (the process generating the data probably exited). To alleviate any confusion, I do not want to break from the loop if even 1 of the rsync processes succeed.

Is there a succinct way to do that in bash?

mgilson
  • 300,191
  • 65
  • 633
  • 696
  • Exit status 12 appears to be related to errors in the protocol data stream. Do you want to exit the loop on *any* nonzero exit status (from all the `rsync` processes)? – ezod Oct 19 '12 at 19:11
  • @ezod -- I think any non-zero exit status is good enough for me to break on (but only if they are all non-zero). – mgilson Oct 19 '12 at 19:13

3 Answers3

2

You could do it by adding up the return values, so that if they all return 12, the sum is 48:

while :
do
    rc=0
    rsync -avz --remove-source-files -e ssh me@remote:path/to/foo* ./
    let rc+=$?

    rsync -avz --remove-source-files -e ssh me@remote:path/to/bar* ./
    let rc+=$?

    rsync -avz --remove-source-files -e ssh me@remote:path/to/baz* ./
    let rc+=$?

    rsync -avz --remove-source-files -e ssh me@remote:path/to/qux* ./
    let rc+=$?

    if [[ $rc == 48 ]]; then  # 48 = 4 * 12
         break;
    fi

    sleep 900 #wait 15 minutes, try again
done

Note that this may suffer if you get another combination of return codes sums to 48, i.e. 0 + 0 + 12 + 36

imp25
  • 2,327
  • 16
  • 23
  • I thought about something like this. I don't want to rely on it actually using exit status 12 though ... This is more a boolean sort of thing. I suppose I could do something like `rsync ... || let rc+=1` and then `if [[ $rc -eq 4 ]]; then break; fi` ... – mgilson Oct 19 '12 at 19:16
  • good idea, but to avoid having to worry about other combinations, why not just append the $rc values to each other and then use `case $rc in *[1-9]* ) print "found non-zero exit code=$rc, can't continue" ; exit 1 ;; esac`. Good luck to all. – shellter Oct 19 '12 at 19:22
  • Just add them and break if the sum is non-zero. – chepner Oct 19 '12 at 19:26
  • @shellter -- No, I only want to exit if they *all* fail. If 1 fails, I want to continue. – mgilson Oct 19 '12 at 19:29
  • 1
    ok, then invert the test. `case $rc in *0* ) echo "at least one did not fail, keep running" ;; * ) print "no zeros found, must have all failed, quiting" ; exit 1 ;; esac` . Good luck to all. – shellter Oct 19 '12 at 19:55
1

Inspired by the other answers, I think this is the cleanest way I can do it so far ...

while :
do
    do_continue=0

    rsync -avz --remove-source-files -e ssh me@remote:path/to/foo* ./ && do_continue=1
    rsync -avz --remove-source-files -e ssh me@remote:path/to/bar* ./ && do_continue=1
    rsync -avz --remove-source-files -e ssh me@remote:path/to/baz* ./ && do_continue=1
    rsync -avz --remove-source-files -e ssh me@remote:path/to/qux* ./ && do_continue=1

    if [[ $do_continue == 0 ]]; then 
       break
    fi

    sleep 900 #wait 15 minutes, try again
done

which could be refactored more to remove the break statement and associated conditional test:

do_continue=1
while [ do_continue -eq 1 ]; do
    do_continue=0
    rsync -avz --remove-source-files -e ssh me@remote:path/to/foo* ./ && do_continue=1
    #...
    sleep 900
done
mgilson
  • 300,191
  • 65
  • 633
  • 696
0

This way counts the number of fails due to no file.

while :
do
    nofile=0

    rsync -avz --remove-source-files -e ssh me@remote:path/to/foo* ./
    (( $? == 12 )) && let nofile++

    rsync -avz --remove-source-files -e ssh me@remote:path/to/bar* ./
    (( $? == 12 )) && let nofile++

    rsync -avz --remove-source-files -e ssh me@remote:path/to/baz* ./
    (( $? == 12 )) && let nofile++

    rsync -avz --remove-source-files -e ssh me@remote:path/to/qux* ./
    (( $? == 12 )) && let nofile++

    # if all failed due to "no files", break the loop
    if (( $nofile == 4 )); then break; fi

    sleep 900 #wait 15 minutes, try again
done
mgilson
  • 300,191
  • 65
  • 633
  • 696
doubleDown
  • 8,048
  • 1
  • 32
  • 48