How to get the "result" from a contextmanager using 'with'

Question

I found a demo of a file locking class (here: https://groups.google.com/forum/#!topic/pug-pe/mQr7KX-cenU), but I don't quite understand the mechanics of using it.

@contextmanager
def FileLock(lock_file):
    if os.path.exists(lock_file):
        print 'Only one script can run at once. '\
              'Script is locked with %s' % lock_file
        sys.exit(-1)
    else:
        open(lock_file, 'w').write("1")
        try:
            yield
        finally:
            os.remove(lock_file)

I believe this says "if the file that is passed in does not exist, open it. When you are done, delete it.

The demonstrated use is:

with FileLock('/tmp/my_script.lock'):
    print "I am here and I am sleeping for 10 sec..."
    time.sleep(10)

and it works correctly - if I run the script once I see "I am here and I am sleeping for 10 sec...", and if I run it again within that 10 seconds, I see "Only one script can run at once. Script is locked with /tmp/my_script.lock". However, to use a file lock, one would typically want to "wait until the lock is obtained" before doing something. However, the 'sys.exit()' seems to prevent that. It seems like I would want to wrap the 'with' in a while loop somehow? Something like:

while fileIsLocked:
    with FileLock('/tmp/my_script.lock'): # try to get the lock
        print "I am here and I am sleeping for 10 sec..."
        time.sleep(10)

but I don't understand how to get a return value from FileLock. Can someone please explain how to do this?

Read the documentation of the `with` statement, the form is `with foo() as bar`. — l4mpi, Apr 23 '14 at 20:58
Your example code is not designed to be used this way. The `FileLock` decorated function does not support "waiting", but provides only a very basic functionality. If you need something more powerful, you might need a more complex solution. — Ferdinand Beyer, Apr 23 '14 at 20:59
Don't even try to use "busy waiting" for this scenario. Consider using a cross-process lock/mutex as provided by the `multiprocessing` module: https://docs.python.org/2/library/multiprocessing.html#synchronization-between-processes — Ferdinand Beyer, Apr 23 '14 at 21:01

score 3 · Accepted Answer · answered Apr 23 '14 at 21:04

You should use something like the following:

@contextmanager
def FileLock(lock_file):
    while os.path.exists(lock_file):
        print 'Only one script can run at once. '\
              'Script is locked with %s' % lock_file
        time.sleep(1)
    open(lock_file, 'w').write("1")
    try:
        yield
    finally:
        os.remove(lock_file)

This directly answers your declared need. OTOH the method is very buggy because there is explicit race condition between checking file existence and opening one. A more stable approach should use O_EXCL to ensure file didn't exist during creation, or flock to use locking on the file contents, not its presence. Also, some kernel-level IPC can be used (Posix semaphores, SysV semaphores, etc.)

score 0 · Answer 2 · answered Apr 23 '14 at 21:27

I have accepted Netch's answer, as it both demonstrates the solution to my 'declared' need, and recommends a better solution. I have posted the use of flock here for completeness.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import fcntl
import time

try:
  f = open('/tmp/locktest', 'w')
  fcntl.flock(f, fcntl.LOCK_EX) # Get an exclusive lock - this will block until it gets the lock
  print "Sleeping..."
  time.sleep(10)
except IOError:
  print("can't immediately write-lock the file ($!), blocking ...")
else:
  print("No error")

  print("End of file")

How to get the "result" from a contextmanager using 'with'

2 Answers2