Dynamically setting properties with metaclass

Question

I'm facing a bug when creating a metaclass that dynamically creates classes with properties given a config file.

In more detail, I expect to dynamically create classes with some properties (regular setter and a getter that runs the result of a custom function on that attribute). So I created a Factory class to register classes at runtime and a metaclass to set the properties given the config dict.

from copy import deepcopy
from typing import Any, Callable, Dict, Tuple


class Meta(type):
    def __new__(
        cls,
        clsname: str,
        bases: Tuple,
        attrs: Dict[str, Any],
        fields: Dict[str, Callable] = None,
    ) -> object:
        if fields is not None:
            for field, field_fn in fields.items():
                attrs[f"_{field}"] = None
                attrs[f"{field}"] = property(
                    fget=lambda self: deepcopy(field_fn)(getattr(self, f"_{field}")),
                    fset=lambda self, value: setattr(self, f"_{field}", value),
                )
        return super().__new__(cls, clsname, bases, attrs)


class Factory:
    registry = {}

    @classmethod
    def register(
        cls,
        name: str,
        cfg: Dict[str, Callable],
    ) -> None:
        class ConfigurableClass(metaclass=Meta, fields=cfg):
            pass

        Factory.registry[name] = ConfigurableClass

    @classmethod
    def make(cls, name: str):
        return cls.registry[name]()


if __name__ == "__main__":
    Factory.register("foo", {"a": lambda x: x + 1, "b": lambda x: x - 1})
    obj = Factory.make("foo")
    obj.a = 5
    obj.b = 5
    print(obj.a, obj.b)
    # Expected 6 and 4 but get 4 and 4 instead

However, for some reason, the function of the last dict key gets registered for all properties. I even tried to add a deepcopy in there.

Any help is appreciated, thanks.

Answer 1

the problem is that you have a lambda function for the getter, so the value for field_fn is lazy-evaluated when the getter for a property is first called; therefore, all properties will return the value for the local var field_fn that is set in the last loop iteration, which explains the weird behavior you were noticing.

You can solve this by creating an outer function that wraps the property getter in a function as shown below, and pass arguments to the function, so that the local var field_fn for each property is explicitly set. This way, each property will use the local field_fn value that is explicitly bound to it.

from typing import Any, Callable, Dict, Tuple


class Meta(type):
    def __new__(
        cls,
        clsname: str,
        bases: Tuple,
        attrs: Dict[str, Any],
        fields: Dict[str, Callable] = None,
    ) -> object:
        if fields is not None:
            for field, field_fn in fields.items():

                # the problem is that you have a lambda function, so the `field_fn`
                # is only evaluated when the `getter` is first called. Since the
                # `field_fn` is lazy evaluated in this way, the value of `field_fn`
                # from the last loop iteration is used instead. You can solve this
                # by creating a function that binds the local `field_fn` for each
                # iteration explicitly.
                def getter_for_field(field_fn: Callable):
                    # lambda function here will use `field_fn` in function
                    # locals, *not* the one which is set in each loop iteration.
                    return lambda self: field_fn(getattr(self, under_f))

                under_f = f'_{field}'

                attrs[under_f] = None
                attrs[field] = property(
                        ## Modified
                        fget=getter_for_field(field_fn),
                        ## End
                        fset=lambda self, value: setattr(self, under_f, value),
                    )

        return super().__new__(cls, clsname, bases, attrs)


class Factory:
    registry = {}

    @classmethod
    def register(
        cls,
        name: str,
        cfg: Dict[str, Callable],
    ) -> None:
        class ConfigurableClass(metaclass=Meta, fields=cfg):
            pass

        Factory.registry[name] = ConfigurableClass

    @classmethod
    def make(cls, name: str):
        return cls.registry[name]()


if __name__ == "__main__":
    Factory.register("foo", {"a": lambda x: x + 1, "b": lambda x: x - 1})
    obj = Factory.make("foo")
    obj.a = 5
    obj.b = 5
    print(obj.a, obj.b)
    # Expected 6 and 4 but get 4 and 4 instead

Alternatively, you can also use the below shorthand instead of creating an outer function. This adds a new parameter to the lambda which binds the field_fn to the lambda function locals. Since the usage is as a property getter, invoked like obj.a for example, the parameter fn will use the default value which is set in any case, which is the desired behavior here.

lambda self, fn=field_fn: fn(getattr(self, under_f))

Output:

6 4