How to test if Cython property is generator?

Question

In IPython, I can see that a property of a Cython class is a generator by simply defining it and then calling:

%%cython
cdef class SomeCls:
    property x:
        def __get__(self):
            yield 1

The call looks like

SomeCls().x
# prints <generator at 0x102f61ee8>

I am having trouble testing if that property is a generator:

import types
print(isinstance(SomeCls().x, types.GeneratorType))
# prints False

import inspect
print(inspect.isgeneratorfunction(SomeCls.x))
# prints False

How can I determine whether a property of a Cython class is a generator?

Testing against `types.GeneratorType` works just fine when _not_ using cython... — Eric, Nov 21 '18 at 21:00
Don't be picky about where you get help! As an experienced poster, I often answer with short comments. For a regular answer I spend more time, providing working examples and explanations. — hpaulj, Nov 21 '18 at 21:15
This _doesn't_ fix the problem, but Cython now supports the standard `@property` syntax (which I think is now preferred since it matches the rest of Python). It behaves exactly the same though with this. — DavidW, Nov 21 '18 at 22:37

score 2 · Accepted Answer · answered Nov 22 '18 at 06:44

Why doesn't the usual way work?

First, as you already probably know, there is no difference between inspect.isgeneratorfunction(...) and isinstance(..., types.GeneratorType) - the inspect-module just calls isinstance(..., types.GeneratorType).

On the other hand, types.GeneratorType is defined as

def _g():
    yield 1
GeneratorType = type(_g())

CPython uses PyGenObject (here code, here documentation) for generators, there is no fancy logic for the comparison as for some ABC-classes, so the isinstance will boil down to comparing the C-object types.

However, Cython returns a __pyx_CoroutineObject for generators (just check the cythonized code to see)

typedef PyObject *(*__pyx_coroutine_body_t)(PyObject *, PyThreadState *, PyObject *);
typedef struct {
    PyObject_HEAD
    __pyx_coroutine_body_t body;
    PyObject *closure;
    ...
    int resume_label;
    char is_running;
} __pyx_CoroutineObject;

which has nothing to do with PyGenObject as far as isinstanceis concerned - it doesn't really care whether generator is in the name of the type (but for us humans it can be really puzzling, because type(obj) says "generator").

So you will have to roll out your own version of isgenerator, which takes also Cython-"generators" into account. There are many ways, for example

%%cython
def _f():
    yield 1
CyGeneratorType = type(_f())   
def iscygenerator(o):
    return isinstance(o, CyGeneratorType)

and now:

import inspect   
def isgenerator(o):
    return inspect.isgenerator(o)  or iscygenerator(o)

isgenerator(SomeCls().x)          #  True
iscygenerator(SomeCls().x)        #  True
inspect.isgenerator(SomeCls().x)  #  False

This answered the question in the MWE that I posted, but unfortunately it didn't solve my problem (determining the type of [this Spacy attribute](https://github.com/explosion/spaCy/blob/develop/spacy/tokens/token.pyx#L456)). However, I was able to use your example to set `SpacyGeneratorType = type(doc.subtree)` and then test against that type, like you showed. This mostly teaches me how little I know about types in Cython. Thanks for your help! — user1717828, Nov 27 '18 at 18:45

How to test if Cython property is generator?

1 Answers1