Thread Safe Python Property/Attribute?

Question

I have code like the following:

class SomeSharedData(object):
    def __init__(self):
        self._lock = RLock()
        self._errors = 0

    @property
    def errors(self):
        with self._lock:
        return self._errors

    @errors.setter
    def errors(self, value):
        with self._lock:
            self._errors = value

Except with more properties like errors. My goal here is ease-of-use, and not efficiency, so the excessive locking is fine.

Is there a more concise way of defining these properties?

The best I've come up with so far is this:

class thread_safe_property(object):
    def __init__(self, name=None):
        self.name = name

    def __get__(self, obj, objtype):
        with obj._lock:
            return getattr(obj, self.name)

    def __set__(self, obj, val):
        with obj._lock:
            setattr(obj, self.name, val)

class SomeSharedData(object):
    def __init__(self):
        self._lock = RLock()
        self._errors = 0

    errors = thread_safe_property('_errors')

Ideas? Better Ways?

Both the original code and the new approach suffer from possible race conditions on statements like data.errors += 1, but I rarely need to perform those operations so I add workarounds where needed.

Thank you!

Good catch. That was a re-formatting error for StackOverflow on my part :P — morrog, Aug 06 '11 at 02:35

Duncan · Accepted Answer · 2011-08-05T09:41:12.693

You possibly need to think a bit harder about exactly what it means to be thread safe. Consider if you wrote this code instead:

class SomeSharedData(object):
    def __init__(self):
        self.errors = 0

This code is exactly as thread-safe as the code that you posted. Assigning a value to an attribute is thread safe in Python: the value is always assigned; it may or may not be overwritten by another value assigned from another thread, but you always get one value or the other, never a mix of the two. Likewise accessing the attribute gives you the current value.

Where your code breaks is that as you said with either your original or the simplified version, a line such as:

shared.errors += 1

is not thread safe, but that's the whole point of making you code safe, those are the things you need to watch out for, not the simple get/set.

Answering the question in the comment:

Simple assignment in Python is simply rebinding a name (not making a copy) and is guaranteed atomic; you get one value or the other. However assigning to an attribute (or subscripted variable) could be overridden as in your property above. In that case attribute assignment could potentially break. So the answer is that attribute assignment is usually safe but not if it has been over-ridden with a property or setattr or similar.

Also, if the old value that is being replaced is a Python class with a destructor, or something that contains a Python class with a destructor then the destructor code could run in the middle of the assignment. Python will still ensure its own data structures don't get corrupted (so you should never get a segfault) but it won't do the same for your own. The obvious fix for this is never ever to define del on any of your classes.

Thank you for the answer! Quick question: In Python, if two threads write to an object's attribute at will, and a third thread reads that same attribute, will the third thread get back sane results? Either what thread 1 wrote or thread 2, but nothing in-between (byte mixing or garbage)? I'm refactoring old code, and I think I was being too paranoid the first time I wrote it. I don't care about multi-core cache issues or other things that might make the third thread get old values, just as long as the values it does get are sane. — morrog, Aug 05 '11 at 09:08

Thread Safe Python Property/Attribute?

1 Answers1