How to expose a function returning a C++ object to Python using Cython?

Question

I'm building C++<->Python bindings using Cython, and I cannot find how to return a C++ object from a Python method.

More specifically, when compiling peak_detection_.pyx, shown below, I get

peak_detection_.pyx:35:36: Cannot convert 'vector[Peak]' to Python object

for the last lines

def getPeaks(self,data):
    return self.thisptr.getPeaks(data)

I understand the error, but I would not mind some help/pointers about how to fix it.

peak_detection.hpp

#ifndef PEAKDETECTION_H
#define PEAKDETECTION_H

#include <string>
#include <map>
#include <vector>

#include "peak.hpp"


class PeakDetection
{
    public:
        PeakDetection(std::map<std::string, std::string> config);
        std::vector<Peak> getPeaks(std::vector<float> &data);

    private:
        float _threshold;               
};

#endif

peak_detection.cpp

#include <iostream>
#include <string>

#include "peak.hpp"
#include "peak_detection.hpp"


using namespace std;


PeakDetection::PeakDetection(map<string, string> config)
{   
    _threshold = stof(config["_threshold"]);
}

vector<Peak> PeakDetection::getPeaks(vector<float> &data){

    Peak peak1 = Peak(10,1);
    Peak peak2 = Peak(20,2);

    vector<Peak> test;
    test.push_back(peak1);
    test.push_back(peak2);

    return test;
}

peak.hpp

#ifndef PEAK_H
#define PEAK_H

class Peak {
    public:
        float freq;
        float mag;

        Peak() : freq(), mag() {}
        Peak(float f, float m) : freq(f), mag(m) {}
};

#endif

peak_detection_.pyx

# distutils: language = c++
# distutils: sources = peak_detection.cpp

from libcpp.vector cimport vector
from libcpp.map cimport map
from libcpp.string cimport string

cdef extern from "peak.hpp":
    cdef cppclass Peak:
        Peak()

cdef class PyPeak:
    cdef Peak *thisptr
    def __cinit__(self):
        self.thisptr = new Peak()
    def __dealloc__(self):
        del self.thisptr

cdef extern from "peak_detection.hpp":
    cdef cppclass PeakDetection:
        PeakDetection(map[string,string])
        vector[Peak] getPeaks(vector[float])

cdef class PyPeakDetection:
    cdef PeakDetection *thisptr
    def __cinit__(self, map[string,string] config):
        self.thisptr = new PeakDetection(config)
    def __dealloc__(self):
        del self.thisptr
    def getPeaks(self, data):
        return self.thisptr.getPeaks(data)

Isn't there a `cpdef`, or something like that, that makes function visible to both `cython` and `python`? — hpaulj, Nov 06 '15 at 20:07
Indeed, but it does not help apparently: I get the same compilation error. — jul, Nov 09 '15 at 09:27

Simon Gibbons · Accepted Answer · 2015-11-12T14:24:20.027

Your problem here is that cython doesn't know how to automatically convert the C++ object Peak into the python wrapped version PyPeak.

A version which will copy the instances of Peak that getPeaks returns into a list of PyPeak instances would be:

# distutils: language = c++
# distutils: sources = peak_detection.cpp

from libcpp.vector cimport vector
from libcpp.map cimport map
from libcpp.string cimport string

cdef extern from "peak.hpp":
    cdef cppclass Peak:
        Peak()
        Peak(Peak &)
        float freq, mag


cdef class PyPeak:
    cdef Peak *thisptr

    def __cinit__(self):
        self.thisptr = new Peak()

    def __dealloc__(self):
        del self.thisptr

    cdef copy(self, Peak &other):
        del self.thisptr
        self.thisptr = new Peak(other)

    def __repr__(self):
        return "<Peak: freq={0}, mag={1}>".format(self.freq, self.mag)

    property freq:
        def __get__(self): return self.thisptr.freq
        def __set__(self, freq): self.thisptr.freq = freq

    property mag:
        def __get__(self): return self.thisptr.mag
        def __set__(self, mag): self.thisptr.mag = mag


cdef extern from "peak_detection.hpp":
    cdef cppclass PeakDetection:
        PeakDetection(map[string,string])
        vector[Peak] getPeaks(vector[float])

cdef class PyPeakDetection:
    cdef PeakDetection *thisptr

    def __cinit__(self, map[string,string] config):
        self.thisptr = new PeakDetection(config)

    def __dealloc__(self):
        del self.thisptr

    def getPeaks(self, data):
        cdef Peak peak
        cdef PyPeak new_peak
        cdef vector[Peak] peaks = self.thisptr.getPeaks(data)

        retval = []

        for peak in peaks:
            new_peak = PyPeak()
            new_peak.copy(peak)
            retval.append(new_peak)

        return retval

Once compiled and run we get the expected output:

In [1]: import peak_detection_

In [2]: print peak_detection_.PyPeakDetection({"_threshold" : "0.01"}).getPeaks([1,2,3])
[<Peak: freq=10.0, mag=1.0>, <Peak: freq=20.0, mag=2.0>]

Thanks Simon. Do I have to copy the Peak instances? There is no way to get a reference or to define an interface to the C++ object? — jul, Nov 12 '15 at 13:51
The issue is that when the you leave the python version of `getPeaks` the vector will go out of scope and will deallocate all of the `Peak` instances that are part of it. A way of getting around this would be for `getPeaks` to return a vector of *pointers* to `Peak` instances which you can then assign to the `thisptr` of the `PyPeak` instances. — Simon Gibbons, Nov 12 '15 at 14:15
But returning a vector of pointers would lead to dangling pointers, right? — jul, Nov 12 '15 at 14:56
Your `PyPeak` instance has a `__dealloc__` method exactly for this. When the reference count on each of those instances goes to zero then it will deallocate it's pointer to the `Peak` object. — Simon Gibbons, Nov 12 '15 at 15:00

How to expose a function returning a C++ object to Python using Cython?

1 Answers1

Linked