Passing list of lists (or array of arrays) from python to C using swig

Question

I've been searching a while but not able to find an answer even if the problem is pretty simple. In python I declare a list of N places whose elements are list itself:

list_of_list = []
for i in range(N):
    list_of_list.append([])

I then want to pass this list of list to my C extension and fill it /read it. As an example, in C:

void * fill_list_of_list (PyObject *args){
    int ok;
    PyObject *list_of_list;
    int i, N;
    
    ok = ( PyArg_ParseTuple(args, "iO", &N, &list_of_list));
    
    for (i=0; i < N; i++){
       /*would like to set, for each sublist, its first element to zero*/
       PyList_SetItem( PyList_GetItem(list_of_list,i), 0 , PyFloat_FromDouble(0.) );
    }
 }

From python then I would like the following code:

print(list_of_list)
fill_list_of_list((N, list_of_list))
print(list_of_list)

to output (es. for N = 3):

[[], [], []]
[[0.], [0.], [0.]]

but instead I get a segmentation fault.

What am I doing wrong? Note that the problem is really with the list of lists because in similar way I can perfectly handle list of float. Similarly with numpy array of floats versus numpy array of differently sized arrays.

Your function doesn't return anything? Maybe that's a problem. — DavidW, Mar 26 '21 at 08:31

QMetaObject · Answer 1 · 2021-04-07T22:39:36.113

PyList_SetItem basically replaces an item in a list, but your list is empty so there isn't anything to replace. The CPython API should set an exception when you attempt this. You may want to try using PyList_Append, instead. I also highly recommend you include error-checking around the CPython functions.

I'm not sure if this is the only issue, though, as you'd need to provide a minimal, reproducible example (your code seems to have some errors such as the unpaired ( next to PyArg_ParseTuple and an undeclared variable timestamps). I'm also not familiar with using SWIG, so consider this a partial answer as I may have missed something more specific to SWIG.

EDIT
As per the comments, here's a working example using PyList_Append. Note that I'm only using the CPython API (no SWIG). You must also edit the path to your Python module file if you want to build and run this example for yourself.

/* main.cpp */
#include <iostream>
#include <cstdio>

#define PY_SSIZE_T_CLEAN
#include <Python.h>

#define PYTHONFILE "Path\\To\\Python\\File\\test.py" // <-- Change this to the location  
                                                     //      of your Python module file

extern "C" {
    static PyObject *Foo_fillListOfLists(PyObject *, PyObject *);
}

static PyMethodDef FooMethods[] =
{
    {
        "fill_list_of_lists",
        Foo_fillListOfLists,
        METH_VARARGS,
        "Fills list of lists with zeros."
    },
    { NULL, NULL, 0, NULL }
};

static struct PyModuleDef FooModule =
{
    PyModuleDef_HEAD_INIT,
    "foo",
    NULL,
    -1,
    FooMethods
};

PyMODINIT_FUNC PyInit_Foo(void)
{
    return PyModule_Create(&FooModule);
}

void RunPython()
{
    FILE *fp = NULL;

    if (fopen_s(&fp, PYTHONFILE, "rb"))
        return;

    if (PyImport_AppendInittab("foo", PyInit_Foo) == -1) {
        std::cout << "ERROR: Could not extend built-in modules table\n";
        fclose(fp);
        return;
    }

    Py_Initialize();
    PyRun_SimpleFile(fp, PYTHONFILE);
    Py_FinalizeEx();

    fclose(fp);
}

int main(int argc, char *argv[])
{
    RunPython();
    return 0;
}

PyObject *Foo_fillListOfLists(PyObject *self, PyObject *args)
{
    PyObject *listOfLists;
    Py_ssize_t listLen;

    if (!PyArg_ParseTuple(args, "O", &listOfLists))
        return NULL;

    if (!PyList_CheckExact(listOfLists)) {
        PyErr_SetString(PyExc_RuntimeError, "Received non-list type object.");
        return NULL;
    }

    listLen = PyList_GET_SIZE(listOfLists);
    for (Py_ssize_t i = 0; i < listLen; ++i) {
        PyObject *listInList = PyList_GET_ITEM(listOfLists, i);

        if (!PyList_CheckExact(listInList)) {
            PyErr_SetString(PyExc_RuntimeError, "Non-list type found in list of lists.");
            return NULL;
        }

        // This is what you need to fix your error.
        if (PyList_Append(listInList, PyFloat_FromDouble(0)))
            return NULL;
    }

    return Py_None;
}

# test.py
import foo

list_of_lists = []
for i in range(3):
    list_of_lists.append([])

print(list_of_lists)
foo.fill_list_of_lists(list_of_lists)
print(list_of_lists)

Output:

>> [[], [], []]
>> [[0.0], [0.0], [0.0]]

I corrected the sintax errors you pointed out. My point is actually that I expected the i-th item of listof list to be a pointer to a list object, this is way I'm having troubles with list of list ando not with individual lists. Indeeed for the momento I solved it by creating the list of list in C and then returning it — Marcella, Apr 05 '21 at 14:21
Right, but your call to `PyList_SetItem` uses the `list_of_list` elements (which are empty lists) as the list objects. I've reproduced your extended function and can confirm that `PyList_SetItem` in this context will throw a fatal exception. I'll edit my answer with a working example using PyList_Append so you can see what I mean. — QMetaObject, Apr 07 '21 at 22:10

Passing list of lists (or array of arrays) from python to C using swig

1 Answers1