Threading of Periodic Update to Class Member

Question

What is a method for updating a class member in Python while it is still being used by other methods in the class?

I want the rest of the class to continue processing using the old version of the member until it is fully updated and then switch all processing to the new version once the update is complete.

Here is a toy example to illustrate my use case, where self.numbers is the class member that needs safely threaded periodic updating using the logic in updateNumbers(), which I want called in a non-blocking way by runCounter().

from time import sleep, time

class SimpleUpdater(object):
    def __init__(self):
        self.i = 5
        self.numbers = list(range(self.i))
        self.lastUpdate = time()
        self.updateDelta = 10

    def timePast(self):
        now = time()
        delta = self.lastUpdate - now
        return (delta > self.updateDelta)

    def updateNumbers(self):
        print('Starting Update', flush=True)
        self.numbers = list(range(self.i))
        # artificial calculation time
        sleep(2)
        print('Done Updating', flush=True)

    def runCounter(self):
        for j in self.numbers:
            print(j, flush=True)
            sleep(0.5)
        self.i += 1
        if self.timePast:
            ## Spin off this calculation!! (and safely transfer the new value)
            self.updateNumbers()

if __name__ == '__main__':
    S = SimpleUpdater()
    while True:
        S.runCounter()

The desired behavior is that if self.numbers is being iterated on in the loop, it should finish the loop with the old version before switching to the new version.

Answer 1

You could create a new thread for every call to updateNumbers but the more common way to do this type of thing is to have 1 thread running an infinite loop in the background. You should write a method or function with that infinite loop, and that method/function will serve as the target for your background thread. This kind of thread is often a daemon, but it doesn't have to be. I've modified your code to show how this might be done (I've also fixed a few small bugs in your example code).

from time import sleep, time
import threading


class Numbers(object):
    def __init__(self, numbers):
        self.data = numbers
        self.lastUpdate = time()


class SimpleUpdater(object):
    def __init__(self):
        self.i = 5
        self.updateDelta = 5
        self.numbers = Numbers(list(range(self.i)))
        self._startUpdateThread()

    def _startUpdateThread(self):
        # Only call this function once
        update_thread = threading.Thread(target=self._updateLoop)
        update_thread.daemon = True
        update_thread.start()

    def _updateLoop(self):
        print("Staring Update Thread")
        while True:
            self.updateNumbers()
            sleep(.001)

    def updateNumbers(self):
        numbers = self.numbers
        delta = time() - numbers.lastUpdate
        if delta < self.updateDelta:
            return 
        print('Starting Update')
        # artificial calculation time
        sleep(4)
        numbers = Numbers(list(range(self.i)))
        self.numbers = numbers
        print('Done Updating')

    def runCounter(self):
        # Take self.numbers once, then only use local `numbers`.
        numbers = self.numbers
        for j in numbers.data:
            print(j)
            sleep(0.5)
        # do more with numbers
        self.i += 1


if __name__ == '__main__':
    S = SimpleUpdater()
    while True:
        S.runCounter()

Notice that I have not used any locks :). I could get away without using any locks because I only access the numbers attribute of the SimpleUpdater class using atomic operations, in this case just simple assignments. Every assignment to self.numbers associated a new Numbers object with that attribute. Every time you access that attribute you should expect to get a different object, but if you take a local reference to that object at the beginning of a method, ie numbers = self.numbers, numbers will always refer to the same object (till it goes out of scope at the end of the method), even if the background thread updates self.numbers.

The reason I've created a Numbers class is so that i can get and set all "volatile" members in a single (atomic) assignment (the numbers list and lastUpdated value). I know this is a lot to keep track of, and honestly it might be smarter and safer to just use locks :), but I wanted to show you this option as well.

Answer 2

Create a lock to control access to your list:

import threading

def __init__(self, ...):
    # We could use Lock, but RLock is somewhat more intuitive in a few ways that might
    # matter when your requirements change or when you need to debug things.
    self.numberLock = threading.RLock()
    ...

Whenever you need to read the list, hold the lock and save the current list to a local variable. The local variable will not be affected by updates to the instance attribute; use the local variable until you want to check for an updated value:

with self.numberLock:
    numbers = self.numbers
doStuffWith(numbers)

Whenever you need to update the list, hold the lock and replace the list with a new list, without mutating the old list:

with self.numberLock:
    self.numbers = newNumbers

---

Incidentally, I've used camelcase here to match your code, but the Python convention is to use lowercase_with_underscores instead of camelCase for variable and function names.