Using threading.RLock Correctly

Question

I'm creating a utility in Python that reads data in from a file on startup on a separate thread so that the rest of the GUI components can load. The data gets stored into a list and then appended to a combobox. How would I lock the list so that no other method can call the list at the same time it's being used by the def read_employees(self, read_file): method.

This is the best attempt I can come up with.

#left out imports

class MyDialog(wx.Frame):

    def __init__(self, parent, title):
        self.no_resize = wx.DEFAULT_FRAME_STYLE & ~ (wx.RESIZE_BORDER | wx.MAXIMIZE_BOX)
        wx.Frame.__init__(self, parent, title=title, size=(500, 450),style = self.no_resize)

        self.lock = threading.RLock()
        self.empList = []


    def read_employees(self, read_file):

        with open(read_file) as f_obj:
            employees = json.load(f_obj)

        with self.lock:
            self.empList = [empEmail for empEmail in employees.keys()]
            wx.CallAfter(self.emp_selection.Append, self.empList)


    def start_read_thread(self):
        filename = 'employee.json'
        with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
            executor.submit(self.read_employees, filename)

app = wx.App(False)
frame = MyDialog(None, "Crystal Rose")
app.MainLoop()

Is using RLock appropriate here?

soundstripe · Accepted Answer · 2017-11-14T14:08:59.260

I don't know what else you have going on in the app, but I'd recommend taking a look at the wx.CallAfter function. It is thread-safe and can be used to send messages or post events.

import wx
from wx.lib.pubsub import Publisher
import json
from threading import Thread


def update_employee_list(read_file):
    with open(read_file) as f_obj:
        employee_list = json.load(f_obj) # this line should release the GIL so it continues other threads
    # next line sends a thread-safe message to the main event thread
    wx.CallAfter(Publisher().sendMesage, 'updateEmployeeList', employee_list)


class MyDialog(wx.Frame):
    def __init__(self, parent, title):
        self.no_resize = wx.DEFAULT_FRAME_STYLE & ~ (wx.RESIZE_BORDER | wx.MAXIMIZE_BOX)
        wx.Frame.__init__(self, parent, title=title, size=(500, 450),style = self.no_resize)
        self.empList = []
        # subscribe our function to be called when 'updateEmployeeList' messages are received
        Publisher().subscribe(self.updateDisplay, 'updateEmployeeList')

    def updateDisplay(self, employee_list):
        # this assignment should be atomic and thread-safe
        self.empList = employee_list
        # wxPython GUI runs in a single thread, so this is a blocking call
        # if you have many many list items, you may want to modify this method
        # to add one employee at a time to the list to keep it non-blocking.
        self.emp_selection.Append(employee_list)

    def start_read_thread(self):
        filename = 'employee.json'
        t = Thread(target= update_employee_list, args=(filename, ))
        t.start()  # this starts the thread and immediately continues this thread's execution

Update:

Using a with ThreadPoolExecutor blocks because the code is equivalent to:

executor = ThreadPoolExecutor(max_workers=1)
executor.submit(worker_func, args)
executor.shutdown(wait=True)  # <--- wait=True causes Executor to block until all threads complete

You could still use the ThreadPoolExecutor as follows, without the with block. Because you're only :

executor = ThreadPoolExecutor(max_workers=1)
executor.submit(worker_func, args)
executor.shutdown(wait=False)  # <--- threads will still complete, but execution of this thread continues immediately

For more about concurrent futures and Executors, see here for documention.

Thanks for the time and effort to create this. Few things, in the `start_read_thread`,`self.read_employees` should be `update_employee_list(read_file):` and in `def update_employee_list(read_file):`, `'updateEmployeeList'` should be `updateDisplay` Correct? If I wanted to continue with `R.Lock`, have I used it correctly? — , Nov 14 '17 at 09:33
Yes I’ve fixed that bug. In your example, your ThreadExecutor will block until the thread completes, so I wouldn’t say that it is correct. As far as the lock goes, it’s hard to tell if you used it correctly because I only see one usage of it. If you’re going to use a lock it must be used everywhere that you access that variable. That one looks fine although there’s. I need for CallAfter to be in the locked area. — soundstripe, Nov 14 '17 at 12:38
Could you update your answer to explain why `threadpoolexecutor` will block? I was given the suggestion to use `threadpoolexecutor` in this answer here https://stackoverflow.com/questions/47186213/reading-a-file-without-block-main-thead-in-gui?noredirect=1&lq=1. — , Nov 14 '17 at 13:48
The docs say if I use `with`, I don't have to use shutdown. It will function as if `shutdown` is set to True. — , Nov 14 '17 at 14:13
Not exactly. Please read my update more closely. If `wait = True`, the code will block until all threads complete — soundstripe, Nov 14 '17 at 14:21
Sorry about that. I realized my comment was wrong while doing something else. I have another problem, but I'll post it as a separate question. — , Nov 14 '17 at 15:34
I have a follow up question no one cared to answer so far, would love if you can provide an answer. https://stackoverflow.com/questions/47290677/when-to-call-thread-join-in-a-gui-application — , Nov 15 '17 at 01:36

Using threading.RLock Correctly

1 Answers1

Linked