I ran into this problem using boost::python when our C++ code would trigger a python callback. I would occasionally get "GC object already tracked" and the program would terminate.
I was able to attach GDB to the process prior to triggering the error. One interesting thing, in the python code we were wrapping the callback with
a functools partial, which was actually masking where the real error was occuring. After replacing the partial with a simple callable wrapper class. The "GC object already tracked error" no longer popped up, instead I was now just getting a segfault.
In our boost::python wrapper, we had lambda functions to handle a C++ callback and the lambda function captured the boost::python::object callback function. It turned out, for whatever reason, in the destructor for the lambda, it wasn't always properly acquiring the GIL when destroying the boost::python::object which was causing the segfault.
The fix was to not use a lambda function, but instead create a functor that makes sure to acquire the GIL in the destructor prior to calling PyDECREF() on the boost::python::object.
class callback_wrapper
{
public:
callback_wrapper(object cb): _cb(cb), _destroyed(false) {
}
callback_wrapper(const callback_wrapper& other) {
_destroyed = other._destroyed;
Py_INCREF(other._cb.ptr());
_cb = other._cb;
}
~callback_wrapper() {
std::lock_guard<std::recursive_mutex> guard(_mutex);
PyGILState_STATE state = PyGILState_Ensure();
Py_DECREF(_cb.ptr());
PyGILState_Release(state);
_destroyed = true;
}
void operator ()(topic_ptr topic) {
std::lock_guard<std::recursive_mutex> guard(_mutex);
if(_destroyed) {
return;
}
PyGILState_STATE state = PyGILState_Ensure();
try {
_cb(topic);
}
catch(error_already_set) { PyErr_Print(); }
PyGILState_Release(state);
}
object _cb;
std::recursive_mutex _mutex;
bool _destroyed;
};
