0

I'm trying to run only one instance of a program, this is my runner script which is called from crontab: 

#!/bin/bash

exec 9>./lockfile
if ! flock -n 9  ; then
   echo "another instance of $0 is running";
   exit 1
fi

node $(dirname $0)/init.js

The line in crontab is as follows:

*/5 * * * * /bin/bash /path/to/proyect/runner.sh >> /path/to/proyect/logs/output.log 2>> /path/to/proyect/logs/error.log

The weird thing is that after I kill the process the file stays locked

Francisco Rangel
  • 129
  • 1
  • 11
  • 1
    The lock is tied to an individual inode. If someone has run, say, `rm ./lockfile`, then that untethers the inode opened by the file handle from the filesystem. Sound familiar? – Charles Duffy Nov 27 '17 at 18:29
  • 1
    It's also possible that if your cron is being run in a separate filesystem namespace (which tools like systemd support), it could have a different set of working directories than are visible outside of it. This would be trivial to determine with shell access to your system, but trying to track down all the variables via a Q&A-style session is liable to be a bit painful. – Charles Duffy Nov 27 '17 at 18:30
  • 1
    ...also, err, silly question, but are you **positive** that you know what directory is `.` when your cron job runs? I wonder if you have a `/lockfile` that's held, and your various tests outside of cron have been modifying a `lockfile` in a different directory. – Charles Duffy Nov 27 '17 at 18:31
  • Anyhow -- I suggest making sure you're redirecting stderr to somewhere you actually see it, so you can see any error message emitted by `flock` (or by the `exec` command preceding it) during the cron job's execution; and then similarly logging the output of `stat ./lockfile`, and comparing the inode number against what you get outside of cron. – Charles Duffy Nov 27 '17 at 18:33
  • Sorry @CharlesDuffy, didn't understood the part about the inode. I have redirected the output to a file that I'm looking at while debugging all this – Francisco Rangel Nov 27 '17 at 18:36
  • Be sure you're redirecting stderr, not just stdout. For instance, putting the line `exec >/tmp/cron.log 2>&1` just below the shebang will cover both. – Charles Duffy Nov 27 '17 at 18:37
  • re: "inodes" -- good filesystem design for the last many decades has involved having directory entries be links to "inodes", which actually store all the non-name-linked metadata about a file; a hardlink, then, is when a two different dentries point to the same inode (f/e). Anyhow, the question is whether the inode that's locked (not the filename that's used to refer to it!) matches between the two places you're testing. – Charles Duffy Nov 27 '17 at 18:38
  • so, if you run `stat lockfile` on a GNU system, that will contain something like `Inode: 12189698` -- the inode number. If it's the same between `stat lockfile` run both inside and outside the cron job (and the `Device:` field also matches), then we know with certainty that they refer to the same file. – Charles Duffy Nov 27 '17 at 18:39
  • Yeah no, I'm also looking at the stderr, checked what you said about the placement of the lockfile, fixed that, now of course runs the firs time, what I don't really know if killing the process releases the lock or not – Francisco Rangel Nov 27 '17 at 18:41
  • (inodes being separate from dentries is part of why UNIX allows you to delete open files -- the inode's reference count doesn't hit 0 until there are no handles on it left, so the space could stay allocated -- whereas old DOS filesystems didn't/couldn't do that, and modern Windows with NTFS *can* support it but doesn't by default for compatibility). – Charles Duffy Nov 27 '17 at 18:42
  • ok got it (inodes), thanks! useful info. Will check that next then – Francisco Rangel Nov 27 '17 at 18:43
  • btw -- `node "$(dirname "$0")/init.js"` would be slightly safer quoting. (If you're using `#!/bin/bash`, using `$BASH_SOURCE` instead of `$0` is safer than that -- `$0` [isn't particularly trustworthy](http://mywiki.wooledge.org/BashFAQ/028) -- but that's a separate discussion). I'd also suggest an `exec` for that last line, so the bash instance doesn't need to stick around in memory, so `exec node "$(dirname "$0")/init.js"` – Charles Duffy Nov 27 '17 at 18:43

1 Answers1

0

Weirdly enough opening the file in vim and saving it unlocks the file. This only happens when killing the process from the cli with kill -9 pid

Francisco Rangel
  • 129
  • 1
  • 11
  • That's because vim saves files by writing to a new file and renaming over the original -- replacing it with a new inode pointed at by the same original directory entry. Nothing specific to `kill -9`, and not particularly weird (`sed -i` works the same way, as do many other tools; it means that any reader sees only the old version or the new one, never an interim file that's halfway between). That said, why are you interacting with a lockfile in vim? – Charles Duffy Dec 06 '17 at 17:36
  • The thing that's problematic here, btw, is that it means your problem was caused by **something you didn't tell us you were doing in your question**. That means nobody but you could conceivably have answered this, because the question wasn't complete enough to allow the issue to be reproduced. – Charles Duffy Dec 06 '17 at 17:39
  • I did say I was killing the process, I put it in the answer to make it clear that is not happening when the process fails, only when killing it, what I still do not understand is why the file stays locked when the process is no longer running – Francisco Rangel Dec 07 '17 at 16:21
  • Use the `fuser` command to see which processes are still left behind with a handle on the file -- presumably what you started and then killed left behind a child process. But the thing I was objecting to (that you didn't say anything at all about in the question) was opening and saving the lockfile in vim. – Charles Duffy Dec 07 '17 at 16:27
  • Thanks I'll check that. Yeah no, I wasn't doing that in the first place, it wasn't part of my question, just did it because was trying a few things to unlock the file. As you pointed out something else is probably happening, but for now it solves my problem – Francisco Rangel Dec 07 '17 at 16:49