2

I'm writing a multiprocess bash script that runs several processes (continuously), there are some functions I need to launch, every function has a similar form described in man for flock :

function1 () {
    #.... preamble, variable initialization
    (
             flock -x -w 10 200 || exit 1
             # ... my commands ....
    ) 200>/var/lock/.`basename $0`$conf"-rtrr.lock"
}

Each function has its own fd (number) different from other, every function is independent from others .

The desired behavior is that I should be able to run the same function several times in parallel, the only condition is that only one instance described by function and its parameters can run in every moment. It is not a problem if a function execution fails, the parent runs in a inifinite loop and launch it again. For example if we consider this a list of running processes/functions :

OK
function1 a
function1 b
function2 a
function3 b

NO:
function1 a
function1 a
function2 b
function3 b

On every part I specify different lockfile names using something like :

/var/lock/${program_name}-${parameter}-${function_name}.lock 

example lockfile , if called function1 with a :

/var/lock/program-a-function1.lock

The questions :

  1. using same fd number across several processes, the ones launches the same function, do I risk that one child process overwrite the fd mapping of another child ? The risk is that a process may wait for a wrong lock.
  2. Can I use a variable as fd ? For example a number which is a sort of hash of parameter ?
  3. Otherwise, is there a way to not use fd while using flock command ?
  4. Do you think, for this desired behavior, it is better to use simple files to get and release locks : ieg creating file when acquiring, deleting file when releasing and having an if on top to check lock file presence ?
magowiz
  • 59
  • 5

1 Answers1

0
  1. No risk. When a child process opens a file at some filedescriptor slot, it will overshadow the inherited filedescriptor (if one was inherited into that slot)
  2. You can, but it's ill-advised. For max portability, you should use something less than 10. E.g., 9 like in the flock manpage example.
  3. Yes, as the flock manpage describes, but that implies execing a command, which is not suitable for your case.
  4. Sounds needlessly complicated

If I were you, I'd create lockfiles with > whose names would be derived from $0 (scriptname), ${FUNCNAME[0]} (function-name), and $* (concatenation of function arguments) and use them with a small filedescriptor like 9 like in the flock manpage. If you use basename on the scriptname $0, do it once and save the result in a global.

Example code:

#!/bin/bash

script_name="$(basename "$0")"

func1()(
    echo "$BASHPID: Locking $(readlink /dev/fd/9)"
    flock 9 || return 1
    echo "$BASHPID: Locked $(readlink /dev/fd/9)"
    echo "$BASHPID: WORK"
    sleep 1
    echo "$BASHPID: Release $(readlink /dev/fd/9)"
) 9>/var/lock/"$script_name-${FUNCNAME[0]}-$*"


func1 a & 
func1 a & 
func1 a & 
func1 b & 
func1 b & 
func1 b & 

wait

Possible output of the example code:

16993: Locking /run/lock/locks-func1-b
16985: Locking /run/lock/locks-func1-a
16987: Locking /run/lock/locks-func1-a
16995: Locking /run/lock/locks-func1-b
16994: Locking /run/lock/locks-func1-a
16987: Locked /run/lock/locks-func1-a
16987: WORK
16999: Locking /run/lock/locks-func1-b
16995: Locked /run/lock/locks-func1-b
16995: WORK
16987: Release /run/lock/locks-func1-a
16995: Release /run/lock/locks-func1-b
16985: Locked /run/lock/locks-func1-a
16985: WORK
16993: Locked /run/lock/locks-func1-b
16993: WORK
16985: Release /run/lock/locks-func1-a
16993: Release /run/lock/locks-func1-b
16994: Locked /run/lock/locks-func1-a
16994: WORK
16999: Locked /run/lock/locks-func1-b
16999: WORK
16994: Release /run/lock/locks-func1-a
16999: Release /run/lock/locks-func1-b
Petr Skocik
  • 58,047
  • 6
  • 95
  • 142
  • My favourite lock file strategy in this scenario is to use symlinks instead. `ln -s "pid=$$" /path/to/${0##*/}.pid` will fail if the lockfile exists, and is atomic, so you don't risk overwriting the file if two instances of the script/function get launched at the same time. – ghoti Aug 22 '16 at 13:34
  • @ghoti If you forget about writing PIDs into those lockfiles (and PID files are a fundamentally flawed concept), then you're fine. Only one combo of scriptname, function-name, function-args will get the lock at a time and repeated truncation does no harm to a file that's meant to stay empty. – Petr Skocik Aug 22 '16 at 14:00
  • There exists a race condition between the line in your shell script which *creates* the lockfile and the line which *checks for* your lock file. Pid or something else, the point is to have an atomic operation that both reports on the availability of the lock and creates it simultaneously, thus eliminating the race condition. Creating a symlink satisfies that, since `ln -s` will fail if the target already exists. – ghoti Aug 23 '16 at 12:49
  • The script I described (+ now added an example) doesn't check for a lockfile. It opens a lockfile (possibly creating it) and then locks the lockfile. I don't see where there could be a race in there. – Petr Skocik Aug 23 '16 at 13:12
  • @ghoti The reason I don't like PID files is that those are subject to races as at any point in time only the parent of a PID can be certain that PID hasn't been reaped and recycled (although that's kind of a slow race as PID recycling isn't exactly fast). Please correct me if you think I'm wrong. – Petr Skocik Aug 23 '16 at 13:25