1

In a shell script I want to wait for a certain file to appear

inotifywait -m -r -q -e close_write -e moved_to --format '%w/%f' $ARCHIVE_ROOT | while read FILE; do
# Call other programs which process the file like stat or sha1sum
done

I would have assumed the file to be valid while in the handling code. Sadly the file seems to disappear e.g. after being processed by sha1sum. Did I miss something obvious, which is necessary to make the file last?

abergmeier
  • 13,224
  • 13
  • 64
  • 120

1 Answers1

1

Many processes create temporary files and quickly delete them. Such is the nature of progress.

To prevent a file from being deleted while your loop executes, use a hard link:

inotifywait -m -r -q -e close_write -e moved_to --format '%w/%f' $ARCHIVE_ROOT | while read FILE; do
    ln "$FILE" tmpfile$$ || continue
    sha1sum tmpfile$$
    rm tmpfile$$
done

If the file is destroyed before your while loop starts, there is nothing you can do. If, however, it exists at the start of the loop, the use of a hardlink will ensure that it continues to exist as long as you need it.

The hard link with not stop other processes from changing the file. If that is a problem, then a copy (cp) is necessary.

For this to work, the temporary file has to be on the same filesystem as the original.

If this application is security-sensitive, you may want to use mktemp or other such utility to generate the name for the temporary file.

John1024
  • 109,961
  • 14
  • 137
  • 171
  • So `inotifywait` would rather have to return both the path and inode of the file to work securely!? – abergmeier May 18 '14 at 09:14
  • Having the inode would still not solve the problem if the file has already been deleted before the inotifywait script has a chance to run. – John1024 May 19 '14 at 21:08
  • Correct, but then I would assume for the inode to be reffed (by inotify) until all callbacks of inotify were called. – abergmeier May 21 '14 at 07:27