watch .tar file for changes, and if it does, extract, build, and run

Question

I am trying to watch a tar file for changes, and if it does, extract the files, cd into the dir, and build, and then run.

What I have is as follows:

while [ 1 ]
do
    checksum=`md5sum kinect.tar`
    sleep 10
    newsum=`md5sum kinect.tar`
    if [ $newsum -ne $checksum ]
    then
        killall lebron
        tar -zxf kinect.tar
        cd kinect
        make
        ./lebron &
    fi
done

Occasionally I get a [ has too many arguments error on the line with the if, but I'm not sure why. Is what I have OK? Would you suggest an alternative solution?

Got a feeling that this scheme is going to end in tears. 1) You are assuming that in the sleep the complete tar file is going to arrive. 2) What if it does not build? No lebron - whatever that is 3) What if there is a serious bug in the code that compromises your system? Better to do the cksum before the while loop. perodically check ensuring that the deliver system final operation is to do a `mv` and just warn you that the new file has arrived. — Ed Heal, Feb 10 '12 at 19:41

perelman · Answer 1 · 2012-02-11T00:55:28.370

First, you don't quote your variables. Second, -ne is for comparing numbers, not strings. Use != instead:

if [ "$newsum" != "$checksum" ]

As Ed Heal pointed out in his comment, the algorithm only possibly makes sense if the file is being modified atomically. The best way to guarantee that is if it is generated in one place and then moved to kinect.tar. Also, if the file is modified while the previous version is being built, your code will not catch the change, which can be fixed by the following change suggested by Ed Heal:

checksum=`md5sum kinect.tar`
while [ 1 ]
do
    sleep 10
    old_checksum=$checksum
    checksum=`md5sum kinect.tar`
    if [ "$old_checksum" != "$checksum" ]

Really, it would be best if you had a better way for the process writing the .tar file to communicate with your script. For example, the new .tar files are always fully written and then moved to kinect-new.tar which the script checks for and moves to kinect.tar. That avoids both the possibility of reading an incomplete file and the messiness of comparing md5sums. (You could be fancier and use something like inotify-tools to avoid having to explicitly poll, but it's possibly not worth the effort.)

Unfortunately, that does not solve the problem that the build may fail (unless the .tar files you are using are known good releases, in which case it seems unlikely that you would want to check for a new one every 10 seconds). You probably want to make a directory, untar there, and only replace the old version if make succeeds. Something along the lines of this:

tar -zxf kinect.tar
cd kinect
if make
then
    cd ..
    rm -rf kinect.good
    mv kinect kinect.good
    cd kinect.good
    killall lebron
    ./lebron &
fi
cd .. # make sure we always stay in the original directory

How do you propose I change the algorithm? – tekknolagi Feb 11 '12 at 00:23 — tekknolagi, Feb 11 '12 at 00:23

score 1 · Accepted Answer · answered Feb 11 '12 at 04:16

One of your problems is that md5sum returns both the hash and the file name. You just want to capture the hash. You can easily do this using the read builtin with redirection and process substitution.

Also, since you're already using bash as your interpreter you really aught to use [[ ]] over [ ] and the former is a lot more capable.

#!/bin/bash

while true; do
    read checksum _ < <(md5sum kinect.tar)
    sleep 10
    read newsum _ < <(md5sum kinect.tar)
    if [[ $newsum != $checksum ]]; then
        killall lebron
        tar -zxf kinect.tar
        cd kinect
        make
        ./lebron &
    fi
done

Ed Heal · Answer 3 · 2012-02-11T07:55:57.167

This is a reply to @tekknolagi comment above - using this to make the response more readable.

Script to check for an update

checksum=`md5sum kinect.tar` 

    while [ 1 ] 

do 
    sleep 10 
    newsum=`md5sum kinect.tar` 
    if [ "$newsum" != "$checksum" ] 
    then 
       # Inform you that the file has changed (email perhaps).
    fi

I am assuming that you are receiving the tar file via a network (FTP?). If so do the following

1) FTP to a different file name 2) FTP a small size (1 byte) to indicate FTP is complete 3) Modify the above script to check for the file received by step 2. If found, delete it (or better still rename it to enable you to easily recover), remove the old kinect.tar file. move the file from step 1 into place

Or

1) FTP to a different file name 2) Get FTP to delete kinect.tar 3) Get FTP to move file from 1) into place.

watch .tar file for changes, and if it does, extract, build, and run

3 Answers3