2

I'm working with Bash script and meeting such a situation:

one bash script will write things into a file, and the other bash script will read things from the same file.

In this case, is lockfile necessary? I think I don't need to use lockfile because there are only one reading process and only one writing process but I'm not sure.

Bash write.sh:

#!/bin/bash

echo 'success' > tmp.log


Bash read.sh:

#!/bin/bash
while :
do
    line=$(head -n 1 ./tmp.log)
    if [[ "$line" == "success" ]]; then
        echo 'done'
        break
    else
        sleep 3
    fi
done

BTW, the write.sh could write several key words, such as success, fail etc.

Yves
  • 11,597
  • 17
  • 83
  • 180

2 Answers2

2

While many programmers ignore this, you can potentially run into a problem because writing to the file is not atomic. When the writer does

echo success > tmp.log

it could be split into two (or more) parts: first it writes suc, then it writes cess\n.

If the reader executes between those steps, it might get just suc rather than the whole success line. Using a lockfile would prevent this race condition.

This is unlikely to happen with short writes from a shell echo command, which is why most programmers don't worry about it. However, if the writer is a C program using buffered output, the buffer could be flushed at arbitrary times, which would likely end with a partial line.

Also, since the reader is reading the file from the beginning each time, you don't have to worry about starting the read where the previous one left off.

Another way to do this is for the writer to write into a file with a different name, then rename the file to what the reader is looking for. Renaming is atomic, so you're guaranteed to read all of it or nothing.

Barmar
  • 741,623
  • 53
  • 500
  • 612
  • If so, I think lockfile is not necessary, because the `read.sh` will only `sleep 3` if it reads `suc`. The later loop can get `success`. – Yves Aug 10 '18 at 02:25
  • 1
    Do you even care about the contents of the file? It looks like it would be sufficient to check if `write.sh` has simply *created* an empty file named `./tmp.log`. – chepner Aug 10 '18 at 02:29
  • @chepner I do care about the contents of the file. As you see, the `read.sh` is looking for the key word 'success'. – Yves Aug 10 '18 at 03:14
  • But `write.sh` that you show can never write anything *except* `success`. If that's true, it would be simpler to use the existence, rather than the contents, of the file as an indication of success. – chepner Aug 10 '18 at 03:26
  • @Yves you can have a problem iff one of the possible outputs is an extension of another. If the writer can write `successful partially` then your reader can be mistaken if it reads before the writer is done. – Grisha Levit Aug 10 '18 at 23:29
  • @GrishaLevit Good point, since `$(head -1 ./tmp.log)` discards all trailing newlines of the command, you can't tell whether the file ends with a newline. – Barmar Aug 10 '18 at 23:36
1

At least from your example, it doesn't look like read.sh really cares about what gets written to tmp.log, only that write.sh has created the file. In that case, all read.sh needs to check is that the file exists.

write.sh can simply be

: > tmp.log

and read.sh becomes

until [ -e tmp.log ]; do
  sleep 3
done
echo "done"
chepner
  • 497,756
  • 71
  • 530
  • 681
  • Well, `success` is just an example, it could be `success`, 'fail` etc. This is why I care about if I should lock the file while reading and writing. – Yves Aug 10 '18 at 05:43
  • If it doesn't write `success`, does your script ever exit? Or will the writer overwrite the first line on its next attempt? – chepner Aug 10 '18 at 12:17
  • It sounds like this is something the script writes when it's all done with whatever it's supposed to do, to indicate whether it succeeded or failed. The reader just periodically checks to see if it's done and what the status was. Hopefully these checks are more frequent than the application it's monitoring. – Barmar Aug 10 '18 at 14:59