How to correctly lock Qthreads in pyqt5 using Python3

Question

I am relatively new to python, but was able to get a reasonably useful program to run to crunch a lot of data. I am able to run it over multiple sets of data sequentially using another python script to call the program serially, but I wanted to create a GUI and use multithreading to allow others to use it without knowing all the ins and outs of programming. I created the GUI successfully, and can feed data bidirectionally using signals and slots. What I am having trouble with is creating multiple threads with the same function.

I have done some research and it appears that the function needs to be threadsafe, and unfortunately mine is not because I am using curve_fit() from scipy, which is not threadsafe. So, based on what I have read in this forum and others, I should be using mutex.lock(), but I get the "SystemError: null argument to internal routine" when calling curve_fit()

Here is some sample code to demonstrate what I have done:

    import sip
    sip.setapi('QString', 2)

    import sys, time
    from PyQt5 import QtCore, QtGui, uic, QtWidgets
    from ZthCalculation import ZthObject

    qtCreatorFile = "PyQtZthUI_01.ui" # Enter file here.

    Ui_MainWindow, QtBaseClass = uic.loadUiType(qtCreatorFile)

    #class MyApp(QtGui.QMainWindow, Ui_MainWindow):
    class MyApp(QtWidgets.QMainWindow, Ui_MainWindow):
        def __init__(self):
            super(self.__class__, self).__init__()
            QtWidgets.QMainWindow.__init__(self)
            Ui_MainWindow.__init__(self)
            self.setupUi(self)
            self.RunButton.clicked.connect(self.RunZthTest)
    .
    .
    .
        def RunZthTest(self):
  #create as processes instead of threads???
  #            self.Process1 = QtCore.QProcess()
            self.Thread1 = QtCore.QThread()
            self.obj1 = ZthObject(self.InputWet1.text(), self.InputDry1.text(), self.Output1.text(), self.side1)
            self.obj1.moveToThread(self.Thread1)
            self.Thread1.started.connect(self.obj1.ZthCalculation)
            self.obj1.textBox.connect(self.updateTextBox1)
            self.signal1 = self.obj1.finished.connect(self.Thread1.quit)
            self.Thread1.setObjectName("Thread1")
            self.Thread1.start()
            time.sleep(.1)

            self.Thread2 = QtCore.QThread()
            self.obj2 = ZthObject(self.InputWet2.text(), self.InputDry2.text(), self.Output2.text(), self.side2)
            self.obj2.moveToThread(self.Thread2)
            self.Thread2.started.connect(self.obj2.ZthCalculation)
            self.obj2.textBox.connect(self.updateTextBox2)
            self.signal2 = self.obj2.finished.connect(self.Thread2.quit)
            self.Thread2.setObjectName("Thread2")
            self.Thread2.start()
            time.sleep(.1)

            self.Thread3 = QtCore.QThread()
            self.obj3 = ZthObject(self.InputWet3.text(), self.InputDry3.text(), self.Output3.text(), self.side3)
            self.obj3.moveToThread(self.Thread3)
            self.Thread3.started.connect(self.obj3.ZthCalculation)
            self.obj3.textBox.connect(self.updateTextBox3)
            self.signal3 = self.obj3.finished.connect(self.Thread3.quit)
            self.Thread3.setObjectName("Thread3")
            self.Thread3.start()
    .
    .
    .

    if __name__ == "__main__":
        app = QtWidgets.QApplication(sys.argv)
        window = MyApp()
        window.show()
    #    sys.exit(app.exec_())
        app.exec_()

In another file, I have the main function that I am calling as a thread:

class ZthObject(QtCore.QObject):
    killthread = False
    finished = QtCore.pyqtSignal()
    textBox = QtCore.pyqtSignal(str)
    def __init__(self, wetFilePath, dryFilePath, outFilePath, side, parent=None):
        super(self.__class__, self).__init__()
        self.wetFilePath = wetFilePath
        self.dryFilePath = dryFilePath
        self.outFilePath = outFilePath
        self.side = side
        self.mutex = QtCore.QMutex()
    def cleanup(self):
        ZthObject.killthread = True

#    def ZthCalculation(self, wetFilePath, dryFilePath, outFilePath, side):
    def ZthCalculation(self):
        #calculations here
.
.
.
        print("waypoint2")
        self.mutex.lock()
        popt, pcov = curve_fit(Foster6, timeShort, ZthjcShort, p0 = [Rs, taus])
        self.mutex.unlock()
.
.
.
        self.finished.emit()

I can successfully run the code only calling one thread, but if I call multiple threads, then the output window prints out 'waypoint2' for each thread called, then crashes with the system error I mentioned above.

What am I doing wrong? Do I need to use separate processes instead of Qthreads? Am I misunderstanding how threads work? I want them to operate in independent variable spaces.

score 2 · Answer 1 · answered Jan 28 '17 at 05:10

Using a mutex really only makes something thread safe if all of the other things sharing the internals of the function also respects the mutex. In this case, it won't because, while using a mutex prevents simultaneous calls to curve_fit, you don't know what it is that is thread unsafe about the function, and so you can't be sure that something else won't also use the thread unsafe bit of code at the same time in another thread (e.g. the main thread).

Coupled with the fact that the Python GIL prevents true threading (threading only provides a speed boost in Python if your task is IO bound rather than CPU bound), I would suggest moving to a multiprocess model.

How to correctly lock Qthreads in pyqt5 using Python3

1 Answers1