Cython reify attributes?

Question

I have this (below) code pattern in my project, which is a FOREX client. My question stems from this code review. The outcome of that review resulted in my class definitions that previously derived from tuple now derive form object. This was done to allow for simpler reifying than my suggested idea in the review (and resulted in large performance improvements). The implementation is done in a similar fashion to the class Foo (below).

reify = lambda x: x

class Foo(object):

    def __init__(self, value):
        self._value = value

    def __getattr__(self, item):
        print('Reifying')
        attr = object.__getattribute__(self, '_'+item)
        value = reify(attr)
        setattr(self, item, value)
        return value

Example reifying

>>> foo = Foo(1)
>>> foo.value
Reifying
1
>>> foo.value
1

But also allows for attribute assignment. (After all nothing in python is private)

>>> foo.value = 2
>>> foo.value
2

I would really like to reclaim the safety of tuple instances. Knowing that information from the server can't be accidentally changed and acted upon. (For myself and other's who may choose to use my code)

Ok, so that's context of this Question: How could I implement the above Foo class in cython that presents an immutable instance? inspired by this question.

I naively tried this:

cdef class Bar:
    cdef readonly int value
    cdef int _value

    def __init__(self, value):
        self._value = value

    def __getattr__(self, item):
        attr = object.__getattribute__(self, '_'+item)
        value = reify(attr)
        setattr(self, item, value)
        return value

But quickly found out that __getattr__ is never called because value is initialized with 0

>>> a = Bar(1)
>>> a.value
0
>>>

Answer 1

Let's take a look, why our approach didn't work.

First thing: you cannot access cdef-members from python. That means python doesn't see the cdef int _value, so even if __getattr__ would be called, __getattribute__(self, '_value') would throw.

Second: cdef readonly int value is more than meets the eye.

By declaring a member readonly value you define a property value which has only a getter (and no setter), that you can see in the cythonized C-code.

In the type descriptor (after all Cython creates a C-extension) of the class Bar created by Cython your can find the setters/getters of your class:

static PyTypeObject __pyx_type_10prop_class_Bar = {
   PyVarObject_HEAD_INIT(0, 0)
  "test.Bar", /*tp_name*/
  ....
  __pyx_getsets_10prop_class_Bar, /*tp_getset*/
  ...
};

you could also look up the properties:

static struct PyGetSetDef __pyx_getsets_10prop_class_Bar[] = {
  {(char *)"value", __pyx_getprop_10prop_class_3Bar_value, 0, (char *)0, 0},
  {0, 0, 0, 0, 0}
};

as you can see there is only a getter (called __pyx_getprop_10prop_class_3Bar_value) defined, but not setter defined.

That means, that after the creation an object of class Bar there is already an attribute called value so your __getattr__ is never called with parameter value.

What can be done? I would say, that what you are trying to achieve is a read-only (cached) property. Somewhere along the lines (no Cython involved):

class Foo:
    def __init__(self, value):
        self._value = value
        self._cached_value = None

    @property
    def value(self):
        if self._cached_value is None:
            print("calculating")
            self._cached_value=self._value # evaluate
        return self._cached_value

And now

>>> f=Foo(2)
>>> f.value 
calculating
2
>> f.value # no calculation!
2

Ok, not everything is great:

1. you have some additional checks/indirection compared to your original solution which might worsen the performance slightly, but this is a price to pay for controlling the access to your attribute (either slightly more costs or no control).
2. there is some boilerplate code, but it is not hard to reduce using dynamical nature of python (e.g. decorators).

But this is up to you to make the trade-off.

---

PS: It looks a lot like the code you started with into the code-review. To me, the usage of properties looks more natural (more maintainable, less confusing) than changing "interface" of the objects during the runtime. But there are definitely merits in the proposed usage of __getattr__. You known best where your project is heading, so it is your decision which tools fit best.