Converting Python seqence to arrow Array via C++ API

Question

I'm attempting to investigate how Arrow converts a python list into an equivalent arrow::Array using the C++ API below.

#include <memory>
#include <Python.h>
#include <string>
#include <iostream>
#include <arrow/memory_pool.h>
#include <arrow/python/python_to_arrow.h>


PyObject* clist(void)
{
    PyObject* lst = PyList_New(0);
    PyList_Append(lst, PyLong_FromLong(1));
    PyList_Append(lst, PyLong_FromLong(2));
    PyList_Append(lst, PyLong_FromLong(5));

    return lst;
}

int main()
{
    Py_Initialize();

    PyObject* list = clist();

    std::shared_ptr<arrow::ChunkedArray> carr;

    arrow::py::PyConversionOptions ops;
    ops.from_pandas = false;
    ops.pool = arrow::default_memory_pool(); 
    ConvertPySequence(list, ops, &carr);

    Py_Finalize();

}

The file compiles fine, however I get a segmentation fault at arrow/cpp/src/arrow/python/iterators.h line 44 on PyCheck_Array.

The error in my debugger is EXC_BAD_ACCESS, however when I interrogate it in the debug console it appears to be there in memory:

Any help is appreciated.

Can you come on the dev@arrow.apache.org or user@arrow.apache.org mailing lists? — Wes McKinney, Jun 04 '19 at 14:53

score 1 · Accepted Answer · answered Jun 04 '19 at 23:04

1

You need to initialize the NumPy C API by calling arrow_init_numpy(). See

https://github.com/apache/arrow/blob/master/cpp/src/arrow/python/util/test_main.cc#L24

answered Jun 04 '19 at 23:04

Wes McKinney

101,437
32
142
108

for anybody else looking at this, one can also follow the patterns in `src/arrow/python/python-test.cc` used to test `ConvertPySequence` – clery00 Jun 05 '19 at 09:33

Converting Python seqence to arrow Array via C++ API

1 Answers1