How to release GIL after Py_Initialize?

Question

I am trying to embed python in a GUI application, and so I have a the python interpreter initialized in the main thread of the application, and I may launch python scripts from another one.

I read about PyGILState_Ensure and PyGILState_Release, and I put them around my code which launch python script in my subthread. But the call is blocking, and my code is never run.

It seems that my main thread is not releasing the GIL (checked with PyGILState). The GIL initialization is now done in Py_Initialize, and lot of old doc talk about PyEval_InitThreads and other now deprecated functions.

So with a modern python, how do we release a thread from the GIL after Py_Initialize ?

Are you trying to release the GIL and stop all Python instances? Well, you need the GIL with normal Python. `Py_FinalizeEx()` has a return value (int) or `Py_Finalize` doesn't. https://docs.python.org/3/c-api/init.html - it may be helpful if you explain more of what you're trying to do. Is this an embedded C++ application that calls python scripts? A good starting guide is here BTW: https://docs.python.org/3/extending/embedding.html - the first one is the most basic (just calls a Python script) — Matt, Feb 27 '22 at 16:58

score -1 · Answer 1 · answered Mar 01 '22 at 00:29

The PyGILState functions are designed for code where you don't know if you have the GIL, but you need to have it. Your worker threads should use those functions.

For a case where you need to not have the GIL, you can use PyEval_SaveThread and PyEval_RestoreThread. There are also Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros for when you want to release the GIL for a short block of code, but that doesn't sound like your use case.

A typical use case of PyEval_SaveThread and PyEval_RestoreThread looks like this:

PyThreadState *save;

save = PyEval_SaveThread();
/* do stuff you don't need the GIL for */
PyEval_RestoreThread(save);

where between the PyEval_SaveThread and PyEval_RestoreThread calls, other threads can acquire the GIL.

For your use case, it sounds like you want to release the GIL for much longer than in a typical use case, so you might need to use something longer-lived than a local variable to save the PyThreadState * returned by PyEval_SaveThread. You can call PyEval_RestoreThread later when you finally need the GIL again, perhaps right before you call Py_FinalizeEx.

How to release GIL after Py_Initialize?

1 Answers1