4

I am writting performance aware code in C and wrap the functions parameter with Python C API and ctypes.

I am confused about PyGILState_Ensure. What does it protect? When should I lock or release the GIL.

For example:

extern "C" DLL PyObject * foo(PyObject * capsule, PyObject *pystr) {
    MyClass *pointer = PyCapsule_GetPointer(capsule, "MyClass");
    string s = convert_python_string_to_std_string(pystr);
    auto state = PyGILState_Ensure();
    try {
        vector<string> strs = a_very_computation_function(pointer, s);
        PyGILState_Release(state);
        PyObject *output = convert_std_string_to_python_string_list(strs);
        return output
    }
    catch (std::exception e) {
        PyErr_SetString(PyExc_RuntimeError, e.what());
        PyGILState_Release(state);
        return NULL;
    }
}

In python, ctypes wraps it in such way:

    lib = cdll.LoadLibrary("mydll.dll")
    lib.foo.argtypes = [ py_object, py_object ] #capsule, str
    lib.foo.restype = py_object # [str]

foo function take a python string like "abcd", do some heavy calculation. then return a list of string in python like ["123","qwer","xyz"].

foo function does NOT create any thread.

Since a_very_computation_function can take very long time. I should release the GIL during its exection?

Does reading stuff from PyObject need to be protected from GIL(within "ensure" and "release")?

Does generateing new PyObject need to be protected from GIL?

Does calling PyErr_SetString need to be protected from GIL?

Where should I add PyGILState_Ensure and PyGILState_Release exactly?

worldterminator
  • 2,968
  • 6
  • 33
  • 52
  • It’s a bit weird to use `ctypes` with a C(++) function that is already doing almost all the work of being accessible from Python directly. – Davis Herring Aug 17 '19 at 20:20
  • You say `foo` doesn’t create threads. Is there *anything* multithreaded here? If not, you needn’t do anything. – Davis Herring Aug 17 '19 at 20:22

0 Answers0