5

I've developed code for biopython which I would like to speed up in my own applications by adding .pxd files and compiling with cython. Along the way I added type hints as I thought this would be an improvement (actually I hoped cython would automatically use them, but this has not been the case). Now I find that the type hinted code is incompatible with the cython enhancement I intended. For example, taking the convolve_py.py example from the cython numpy tutorial:

def naive_convolve(f, g):

changing to:

def naive_convolve(f: np.ndarray, g: np.ndarray) -> np.ndarray:

when I then put 

cpdef np.ndarray[np.int_t, ndim=2] naive_convolve(np.ndarray[np.int_t, ndim=2] f, np.ndarray[np.int_t, ndim=2] g)

in the corresponding .pxd file (it works for the not type-hinted case above) I get:

def naive_convolve(f: np.ndarray, g: np.ndarray) -> np.ndarray:
              ^
------------------------------------------------------------

convolve_py.py:4:19: Compiler crash in AnalyseDeclarationsTransform

I have the same results with simpler types in my own code. My understanding is cython should handle 3.x features and the documentation refers to type hinting, is there another approach or option I am missing? My compile line is:

cythonize -3 -i convolve_py.py convolve_py.pxd
robm
  • 1,303
  • 1
  • 16
  • 24
  • 2
    `cython` has its own way of defining 'types'. Don't try to use the Py3 version. That's only useful Py3 documentation and the experimental `typing` module. That py3 typing also has not been implemented for `numpy`. – hpaulj Mar 28 '20 at 17:23
  • 1
    The cython way of adding numpy types, https://cython.readthedocs.io/en/latest/src/userguide/numpy_tutorial.html#adding-types – hpaulj Mar 28 '20 at 18:07
  • 1
    https://stackoverflow.com/questions/38018780/using-type-hints-to-translate-python-to-cython is the only SO with both `[cython] [type-hinting]`. It's dated (3 yrs ago), but I haven't seen any further developments. Nothing new with `numpy` either, https://stackoverflow.com/questions/58041659/numpy-type-hint-that-something-is-both-an-array-and-float32 – hpaulj Mar 28 '20 at 18:21
  • 1
    Related search topics: type-hinting, PEP, mypy, PEP 484, typing module. As far as I know, `mypy` is the main (only?) tool designed to make use of the annotations as spelled out in the PEP. Otherwise the Py3 annotations only serve as a documentation tool. (a few third party packages may use them for specialized purposes.) – hpaulj Mar 28 '20 at 18:31
  • 1
    Searching `cython` I see that 484 is mentioned briefly in the `pure python` mode page, https://cython.readthedocs.io/en/latest/src/tutorial/pure.html#pure-python-mode. Note that this mode does not offer a significant speed gain. And as noted in other places, `np.ndarray` is not a valid 484 annotation. – hpaulj Mar 28 '20 at 18:34
  • 2
    Recent Cython will use type hints, although it's more limited and buggy than the "traditional" ways., and probably doesn't include `np.ndarray[...]` since that's the old way of doing things. There's an [`annotation_typing` directive](https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#compiler-directives) The crash is probably a bug (and should be reported) but I wouldn't expect to be able to set types in a both .pxd and as annotations. Try turning off `annotation_typing` so you're not giving it two sets of conflicting information. – DavidW Mar 28 '20 at 19:50

0 Answers0