Segment fault when creating PyList_New in Python C extention

Question

parser.cpp:

#include <python3.4/Python.h>
#include <fstream>
#include <string>
#include <vector>
#include <iostream>
using namespace std;
extern "C"
{
    PyObject* test(const char* filename)
    {
        size_t LIM = 1000;
        PyObject* result = PyList_New(LIM);
        for (size_t i = 0; i < LIM; i++) {
            PyObject * t = PyList_New(0);
            PyList_Append(t, PyLong_FromLong(1));
            cout << i << "/" << PyList_Size(result) << " " << t << endl;
            if (PyList_SetItem(result, i, t) != 0) {
                cout << "!" << endl;
            }
        }
        Py_INCREF(result);
        return result;
    }
}

compiles by:

g++ -O2 -fPIC -std=c++11 -shared -o parser_c.so parser_c.cpp -lpython3.4m

test.py:

import ctypes
import datetime
p = ctypes.CDLL('./parser_c.so')
p.test.restype = ctypes.py_object
print(p.test("asd"))

Executing python3 test.py is terminated with segmentation error and the last line I've got by debug output is

381/1000 0x7f0996bb5a88

Besides, it's works properly if I don't import datetime in test.py or decrease LIM in parser.cpp.

I can't figure out why that happends.

Can someone help me?

If you import `datetime` before `ctypes`, it works too. – tynn Mar 04 '16 at 20:33 — tynn, Mar 04 '16 at 20:33

score 4 · Accepted Answer · answered Mar 26 '16 at 13:25

The issue is that ctypes releases the global interpreter lock and so calls to Python api functions are undefined

From the documentation:

The Python global interpreter lock is released before calling any function exported by these libraries, and reacquired afterwards.

You need to re-acquire it at the start of your function, and re-release it at the end:

PyObject* test(const char* filename)
{
    PyGILState_STATE gstate = PyGILState_Ensure();

    // The bulk of your code stays the same and goes here ...

    PyGILState_Release(gstate);

    return result;
}

Segment fault when creating PyList_New in Python C extention

1 Answers1