embedding Python in C++ using pybind11, Segmentation fault

Question

I'm trying to embed Python in C++ using pybind11. Embedding got a lot less attention than extension, and useful resources are hard to find.

Here's my naive code

#include "Python.h"
#include "pybind11/pybind11.h"

#include <iostream>

namespace py = pybind11;


void lock_python(PyThreadState* s)
{
    PyEval_RestoreThread(s);
}


PyThreadState* unlock_python()
{
    return PyEval_SaveThread();
}


void run(PyThreadState * _py_thread_state)
{
    if (_py_thread_state) {
        lock_python(_py_thread_state);
    }

    py::object np = py::module::import("numpy");
    auto v = np.attr("sqrt")(py::cast(36.0));

    std::cout << "sqrt(36.0) = " << v.cast<double>() << std::endl;

    py::dict kwargs = py::dict(py::arg("a") = 3);

    if (_py_thread_state) {
        _py_thread_state = unlock_python();
    }
}


int main()
{
    Py_Initialize();
    PyEval_InitThreads();
    PyThreadState * _py_thread_state = unlock_python();

    run(_py_thread_state);

    if (_py_thread_state) {
        lock_python(_py_thread_state);
        delete _py_thread_state;
    }

    return 0;
}

Without the kwargs line, everything looked fine. With it, I got set-fault.

One wild guess is that I need to somehow delete or decref kwargs, which was not used by Python.

Any pointers are appreciated.

The issue is not due to the fact that `kwargs` is not used. In another case I create a `kwargs` and use it in calling a Python function. Segmentation fault still happens. — zpz, Feb 03 '17 at 03:58
Neither is it specific to `dict`. If I create a tuple in a similar way, problem persists. — zpz, Feb 03 '17 at 04:00
The issue seems to be related to the `lock_python`, `unlock_python` business. If I remove them all, problem is gone. However I think I need the lock-unlock, because my real application is certainly much more involved than this example code. — zpz, Feb 03 '17 at 04:07

score 1 · Answer 1 · answered Nov 15 '17 at 14:09

1

If you do not use threads you dont need the locks. The GIL is held as soon as your interpreter is initialized. Further the pybind11 docs state that one should not use the Py_Initialize and Py_Finalize and use the py::scoped_interpreter instead.

answered Nov 15 '17 at 14:09

ManuelSchneid3r

15,850
12
65
103

I'm not actively working on this right now. Will check later. I think 'scoped_interpreter' was added after my question. A recent pybind11 release states they added "support for embedding Python in C++", which sounds very encouraging to me, although embedding was certainly possible before that release. – zpz Nov 20 '17 at 05:49

embedding Python in C++ using pybind11, Segmentation fault

1 Answers1