Python GUI event dispatch thread blocks when other worker thread uses C++ code

Question

I created a simple GUI application and wanted to have prolonged actions to run on a worker thread rather than the event dispatch thread to maintain the GUI responsive.

This works fine as long as the worker thread is running a python function. However, if the function is doing something on a C++ code, the GUI thread blocks as if the prolonged action is performed on it and not on the worker thread!

It looks as if there's some hidden lock that prevent the GUI thread from running in parallel with the worker thread.

I suspected that it might be related to the UI framework in use, so I tried on either TKinter and wxPython and the problem appears on both.

I'm using Visual Studio 2010 and python 2.7.5 on Windows 7

Here's the C++ code:

Note: I tried also non-busy wait with Sleep(timeSec * 1000L), with the same behavior

#include<boost/python.hpp>
#include <ctime>

void busyWait(int timeSec) {
    clock_t beginTime(clock()); 
    while (clock() - beginTime < timeSec * 1000L);
}

using namespace boost::python;

BOOST_PYTHON_MODULE(BusyCpp) {
    def("busyWait", &busyWait, "waits...");
}

And that's the python code

# Set the path to append the DLL to it
from Utils import ToolEnvironment
ToolEnvironment.useToolBinaries()

from threading import Thread
import Tkinter
import BusyCpp


class simpleapp_tk(Tkinter.Tk):

    def __init__(self, parent):
        Tkinter.Tk.__init__(self, parent)

        self.waitPeriod = 5  # seconds

        button1 = Tkinter.Button(self, text=u"Busy C++", command=self.OnBusyCppClick)
        button1.place(x=20, y=20)

        button2 = Tkinter.Button(self, text=u"Busy Python", command=self.OnBusyPyClick)
        button2.place(x=20, y=60)

    def OnBusyCppClick(self):
        t = Thread(target=self.busyWaitCpp)
        t.start()
        print 'Started thread'

    def OnBusyPyClick(self):
        t = Thread(target=self.busyWaitPy)
        t.start()
        print 'Started thread'

   def busyWaitCpp(self):
        BusyCpp.busyWait(self.waitPeriod)
        print 'Done waiting C++'

   def busyWaitPy(self):
        from time import time
        beginTime = time()
        while time() - beginTime < self.waitPeriod:
            pass
        print 'Done waiting python'

if __name__ == "__main__":
    app = simpleapp_tk(None)
    app.title('my application')
    app.mainloop()

When clicking the "Busy Python" button, one can see that the GUI is responsive (the button looks as unclicked) and these printouts appear in this order, where the "Started thread" appears immediately, as expected:

Started thread
Done waiting python

When clicking the "Busy C++" button, one can see that the GUI is not responsive (the button looks as clicked while waiting) and these printouts appear in this order, where both appear only once waiting is over:

Done waiting C++
Started thread

So it's evident that only once the worker thread finished its job, the GUI thread was able to print "Started thread"

Any idea how to overcome this threading issue?

Thanks

cdarke · Answer 1 · 2013-11-12T10:50:36.560

0

"It looks as if there's some hidden lock" Well it is not hidden that well, it is called the Global Interpreter Lock (GIL) and anyone doing multi-threading in Python should be aware of it. https://wiki.python.org/moin/GlobalInterpreterLock

The C++ code can acquire and free the lock itself. See http://docs.python.org/2/c-api/init.html.

edited Nov 12 '13 at 10:50

answered Nov 12 '13 at 09:03

cdarke

42,728
8
80
84

Thanks, it explains it all indeed. I used @ectamur ScopedGILRelease and it worked. – Yalon Lotan Nov 12 '13 at 11:21

score 0 · Answer 2 · answered Nov 12 '13 at 09:09

0

You need to release the GIL when performing long-running computations in extension code called from Python.

You can use the paired macros Py_BEGIN_ALLOW_THREADS Py_BEGIN_ALLOW_THREADS, or use a scoped manager (e.g. from here):

class ScopedGILRelease {
public:
    inline ScopedGILRelease() { m_thread_state = PyEval_SaveThread(); }
    inline ~ScopedGILRelease() {
        PyEval_RestoreThread(m_thread_state);
        m_thread_state = NULL;
    }
private:
    PyThreadState* m_thread_state;
};

answered Nov 12 '13 at 09:09

ecatmur

152,476
27
293
366

That was very helpful. After adding ScopedGILRelease instance to my C++ function, everything worked like charm. – Yalon Lotan Nov 12 '13 at 11:19
@YalonLotan: This [answer](http://stackoverflow.com/a/18648366/1053968) provides a solution that could decouple `busyWait()` from having to be aware of Python. It may be worth considering if you have C++ threads running outside of Python that invoke `busyWait()`. – Tanner Sansbury Nov 12 '13 at 16:17

Python GUI event dispatch thread blocks when other worker thread uses C++ code

2 Answers2