I am writting performance aware code in C and wrap the functions parameter with Python C API and ctypes.
I am confused about PyGILState_Ensure
. What does it protect? When should I lock or release the GIL.
For example:
extern "C" DLL PyObject * foo(PyObject * capsule, PyObject *pystr) {
MyClass *pointer = PyCapsule_GetPointer(capsule, "MyClass");
string s = convert_python_string_to_std_string(pystr);
auto state = PyGILState_Ensure();
try {
vector<string> strs = a_very_computation_function(pointer, s);
PyGILState_Release(state);
PyObject *output = convert_std_string_to_python_string_list(strs);
return output
}
catch (std::exception e) {
PyErr_SetString(PyExc_RuntimeError, e.what());
PyGILState_Release(state);
return NULL;
}
}
In python, ctypes wraps it in such way:
lib = cdll.LoadLibrary("mydll.dll")
lib.foo.argtypes = [ py_object, py_object ] #capsule, str
lib.foo.restype = py_object # [str]
foo function take a python string like "abcd"
, do some heavy calculation.
then return a list of string in python like ["123","qwer","xyz"]
.
foo
function does NOT create any thread.
Since a_very_computation_function
can take very long time. I should release the GIL during its exection?
Does reading stuff from PyObject
need to be protected from GIL(within "ensure" and "release")?
Does generateing new PyObject
need to be protected from GIL?
Does calling PyErr_SetString
need to be protected from GIL?
Where should I add PyGILState_Ensure
and PyGILState_Release
exactly?