3

I am trying work out how to use cython to speed up a calculation involving integration that is done within a class I have defined. I'm trying to understand better how cython works with user-defined python classes. I'd like to understand more about why the error described below occurs.

I have the following cython code in a file called ex.pyx

from libc.math cimport log
from libc.math cimport sqrt
import scipy.integrate as integ


cdef double integrand_outside(double x):
    """Cython: integrand outside the class"""
    cdef double f = log(x) / sqrt(x)
    return f


cdef class CalcSomething(object):

    def integrate_other(self, double a, double b):
        """This does the integral with the integrand outside the class"""
        return integ.quad(integrand_outside, a, b)[0]

    def integrate_self(self, double a, double b):
        """This does the integral with the integrand inside the class"""
        return integ.quad(self._integrand, a, b)[0]

    def integrate_wrap_self(self, double a, double b):
        """This does the integral with the integrand inside the class"""
        return integ.quad(self.wrap_integrand, a, b)[0]

    def wrap_integrand(self, double x):
        """Python method that wraps _integrand"""
        return self._integrand(x)

    cdef double _integrand(self, double x):
        """Cython: integrand inside the class"""
        cdef double f = log(x) / sqrt(x)
        return f

It shows three ways of calling scipy.integrate.quad from within a class

  1. Using a cython integrand function defined outside the class: integrate_other (ok!)
  2. Using a cython integrand function defined inside the class: integrate_self (produces error)
  3. Wrapping the cython integrand function defined inside the class in a python function defined inside the class: integrate_wrap_self (ok!)

The above cython code compiles fine. Now I call each of these integrate methods, e.g.

import ex

calcSomething = ex.CalcSomething()

a = 0.001
b = 0.1

calcSomething.integrate_other(a,b)     # works
calcSomething.integrate_wrap_self(a,b) # works
calcSomething.integrate_self(a,b)      # doesn't work

Here is the traceback:

Traceback (most recent call last):
File "../examples/example.py", line 10, in <module>
print "integrate self =", calcSomething.integrate_self(a,b)   # doesn't work
File "ex.pyx", line 17, in ex.CalcSomething.integrate_self (ex.c:989)
return integ.quad(self._integrand, a, b)[0]
File "/home/alex/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scipy/integrate/quadpack.py", line 281, in quad
retval = _quad(func,a,b,args,full_output,epsabs,epsrel,limit,points)
File "/home/alex/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scipy/integrate/quadpack.py", line 345, in _quad
return _quadpack._qagse(func,a,b,args,full_output,epsabs,epsrel,limit)
File "stringsource", line 30, in cfunc.to_py.__Pyx_CFunc_double____CalcSomething____double___to_py.wrap (ex.c:1560)
TypeError: wrap() takes exactly 2 positional arguments (1 given)

Here are my questions:

  • Why can scipy.integrate be passed as a cython function or a python method (so now with instance as first argument) but not as a cython method? The error: TypeError: wrap() takes exactly 2 positional arguments (1 given) implies the issue is with the instance argument that is passed with the cython method I think?

  • Is this error coming from my misunderstanding of how to do cython, or a limitation of scipy?

  • Is my intended plan of calculating an integral within a class (by calling an integrand function also within the class) a poor solution if I want to speed up via cython? Disclosure: the real code will call GSL integration functions, not scipy.

alexabate
  • 143
  • 2
  • 10
  • What happens with `calcSomething._integrand(.5)` v `calcSomething.wrap_integrand(.5)`? How about `calcSomething._integrand(None, .5)`. I'm wondering if trying to call the function from outside the class is the problem. – hpaulj Oct 01 '15 at 20:35
  • Doing as you suggest: `calcSomething._wrap_integrand(0.5)'` does work, but `calcSomething._integrand(0.5)` does not throwing: `AttributeError: 'ex.CalcSomething' object has no attribute '_integrand'` However, calling `ex.integrand_outside(double x)` produces `AttributeError: 'module' object has no attribute 'integrand_outside'` – alexabate Oct 02 '15 at 00:48
  • Try `cpdef` for functions that you want to call from Python. – hpaulj Oct 02 '15 at 00:57
  • Ok that solved everything! Thanks! – alexabate Oct 02 '15 at 00:59
  • 1
    As a further comment: I think the auto-generation of a `wrap` function around `cdef` functions when you pass then to places expecting a Python function is a fairly new feature with some issues remaining. They might appreciate a bug report. – DavidW Oct 02 '15 at 06:58
  • When passing `cpdef` functions to scipy routines, does the usage of the Python calling mechanism not incur a speed cost? In situations where the speed of the routines is very critical, would it be worthwhile to look for a pure C/C++ solution that avoids scipy all together? – Scipio Mar 25 '18 at 21:12
  • 1
    @scipio yes, the Python calling does incur a speed cost, and yes, a pure C/c++/Fortran solution is probably the best option. Scipy has [started to develop a solution](https://docs.scipy.org/doc/scipy/reference/generated/scipy.LowLevelCallable.html) but it didn't look to be applied everywhere yet. – DavidW Mar 26 '18 at 12:57

1 Answers1

1

From hpaulj's help above: the answer is the _integrand method needs to be declared cpdef not cdef

cpdef double _integrand(self, double x):
    """Cython: integrand inside the class"""
    cdef double f = log(x) / sqrt(x)
    return f
alexabate
  • 143
  • 2
  • 10