returning tuple of list of lists of complex from c++ to python

Question

I have a PyObject* outvar object which is basically constructed as:

//some defs
typedef std::complex<double> Pt;
typedef std::vector<Pt> Pgon;
typedef std::vector<Pgon> Pgons;
PyObject* outvar = PyTuple_New(2);

auto outA = pgons2pylist(pA);
auto outB = pgons2pylist(pB);
PyTuple_SET_ITEM(outvar, 0, outp);
PyTuple_SET_ITEM(outvar, 1, outl);

where pgons2pylist

PyObject* pgons2pylist(const Pgons& data) {
  PyObject* listObj = PyList_New( data.size() );
    for (unsigned int i = 0; i < data.size(); i++) {
        PyList_SET_ITEM(listObj, i, pgon2pylist(data[i]));
    }
    return listObj;
}

and pgon2pylist is:

PyObject* pgon2pylist(const Pgon& data) {
  PyObject* listObj = PyList_New( data.size() );
    for (unsigned int i = 0; i < data.size(); i++) {
        PyList_SET_ITEM(listObj, i, PyComplex_FromDoubles(data[i].real(),data[i].imag()));
    }
    return listObj;
}

I compile it and run it from py file as:

mylib = ctypes.cdll.LoadLibrary('./mylib.so')
out_data = mylib.call(some_args)

but out_data is always an integer! how can I converted to [[complex]]?

Answer 1

I understand from your example that you want to call a C++ function from Python which takes a list of complex numbers and return them.

If you use pybind11 you get the conversion from std::vector and std::complex for free. pybind11 is a header only library.

Here is an example

example.cpp

#include <pybind11/pybind11.h>
#include <pybind11/complex.h>
#include <pybind11/stl.h>

namespace py = pybind11;

std::complex<double> foo(std::complex<double> a)
{
    return a;
}

std::vector<std::complex<double>> foo2(std::vector<std::complex<double>>& v)
{
    return v;
}

PYBIND11_MODULE(samplepy, m) {  
    m.def("foo", &foo, "Return the complex number entered as an argument");
    m.def("foo2", &foo2, "Return the list of complex number entered as an argument");
}

CMakeLists.txt

cmake_minimum_required (VERSION 3.13)
project (samplepy CXX)
set(CMAKE_CXX_STANDARD 17)
file (GLOB_RECURSE SOURCES "*.cpp" "*.h")

if (WIN32)
    find_package(PythonInterp)
    find_package(PythonLibs 3.6 REQUIRED)
    set(PYBIND11_CPP_STANDARD /std:c++latest)
else()
    find_package(PythonLibs 3.6 REQUIRED)
    set(PYBIND11_CPP_STANDARD -std=c++1z)
endif()

add_library (samplepy SHARED ${SOURCES})
target_link_libraries(samplepy PRIVATE ${PYTHON_LIBRARIES})
set_target_properties(samplepy PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}" SUFFIX "${PYTHON_MODULE_EXTENSION}")

include_directories(${CMAKE_SOURCE_DIR}/include)
include_directories(${PYTHON_INCLUDE_DIR})

example.py

import samplepy
z = complex(2, -3)
print(type(samplepy.foo(z)))
print(samplepy.foo(z))

l = [complex(1, -2), complex(3, -4)]
print(type(samplepy.foo2(l)))
print(samplepy.foo2(l))

When I run this I get this on the py console

<class 'complex'>
(2-3j)
<class 'list'>
[(1-2j), (3-4j)]

Answer 2

The way you structured your C code, looks closer to an extension module ([Python 3]: Extending Python with C or C++) rather than a simple .dll. Check [SO]: Pass str as an int array to a Python C extended function (extended using SWIG) (@CristiFati's answer) for a comparison between methods.

Then, as a note you need to specify argtypes and restype for an imported function (this is the exact reason why you get an int). Check [SO]: Python ctypes cdll.LoadLibrary, instantiate an object, execute its method, private variable address truncated for what might happen if you don't.

Also listing [Python 3]: ctypes - A foreign function library for Python page.

Couple of notes about the code:

• Since you use Python C API functions in your .dll, you should call it via ctypes.PyDLL (ctypes.pydll.LoadLibrary)
• Since you're returning a PyObject*, on Python side you should use py_object (check [SO]: How to cast a ctypes pointer to an instance of a Python class for more details)

So, assuming that you have a working .dll, here's how would you use it (posting the code blindly):

mylib = ctypes.pydll.LoadLibrary('./mylib.so')
outvar = ctypes.py_object.in_dll(mylib, "outvar")  # Note that you might have to declare it as extern "C", so its name doesn't get mangled

@EDIT0:

I created a dummy example to test whether everything works.

dll.c:

#include <Python.h>

#if defined(_WIN32)
#  define EXPORT __declspec(dllexport)
#else
#  define EXPORT
#endif


EXPORT PyObject *tp = NULL;
EXPORT int i = 123;
EXPORT char *s = "Gainarie";
EXPORT float f = -3.14;


EXPORT void initTpl() {
    tp = PyTuple_New(2);
    PyTuple_SET_ITEM(tp, 0, PyLong_FromLong(7));
    PyTuple_SET_ITEM(tp, 1, PyLong_FromLong(-9));
}

code.py:

#!/usr/bin/env python3

import sys
import ctypes


def main():
    dll = ctypes.PyDLL("./dll.so")
    i = ctypes.c_int.in_dll(dll, "i")
    s = ctypes.c_char_p.in_dll(dll, "s")
    f = ctypes.c_float.in_dll(dll, "f")
    dll.initTpl()
    tp = ctypes.py_object.in_dll(dll, "tp")

    print(i.value, s.value, f.value, tp.value, type(tp.value))


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

Notes:

• I only tested on Win, as I didn't transfer my files on my Lnx VM, but this shouldn't be a problem
• Since it's for demo purposes only, I didn't care about memory leaks (nor did I check whether Py_XDECREF is necessary)

Output:

e:\Work\Dev\StackOverflow\q054429301>dir /b
code.py
dll.c

e:\Work\Dev\StackOverflow\q054429301>"c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64

e:\Work\Dev\StackOverflow\q054429301>cl /nologo /DDLL /MD /I"c:\Install\x64\Python\Python\03.06.08\include" dll.c  /link /NOLOGO /DLL /LIBPATH:"c:\Install\x64\Python\Python\03.06.08\libs" /OUT:dll.so
dll.c
   Creating library dll.lib and object dll.exp

e:\Work\Dev\StackOverflow\q054429301>dir /b
code.py
dll.c
dll.exp
dll.lib
dll.obj
dll.so

e:\Work\Dev\StackOverflow\q054429301>"e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" code.py
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32

123 b'Gainarie' -3.140000104904175 (7, -9) <class 'tuple'>