Python - using metaclass & class attributes in object instances

Question

I have a number of classes that inherit a common one. I need the parent class to keep track of a bunch of dependencies/relationships that are defined at the class level. Something like:

class Meta(type):
    ALLDEPENDENCIES = {}
    def __new__(meta, name, bases, attrs):
        if "DEPENDENCIES" in attrs.keys():
            for key, value in attrs.items():
                if key == "DEPENDENCIES":
                    meta.ALLDEPENDENCIES.update(attrs["DEPENDENCIES"])
        return type.__new__(meta, name, bases, attrs)

class DataTable(DataFrameWrapper, metaclass=Meta):
    pass

class Foo(DataTable):
    DEPENDENCIES = {"a":1}

class Bar(DataTable):
    DEPENDENCIES = {"b":2}

So essentially, as I create new classes (Foo, Bar, Baz...) each of them has a dictionary. I need to merge the info from each dictionary. So I'm using the metaclass, as shown above. Each class as an DEPENDENCIES attribute, and I'm gathering all of those into the ALLDEPENDENCIES attribute defined in the metaclass.

If I do this, it seems to work alright:

import Foo, Bar
print(Foo.ALLDEPENDENCIES)
>> {"a":1, "b":2}
print(Bar.ALLDEPENDENCIES)
>> {"a":1, "b":2}

However, when working if obj instances, the ALLDEPENDENCIES attributes is missing:

f = Foo()
b = Bar()
print(f.ALLDEPENDENCIES)
print(b.ALLDEPENDENCIES)

Attribute error - there is no ALLDEPENDENCIES.

I thought that the class attribute defined in the metaclass would be accessible from self.myattribute in the instances, just like DEPENDENCIES is. What am I doing wrong?

It's not exactly a dupe, but there's a nice (although a bit old) discussion of this here that may help: [methods of metaclasses on class instances](https://stackoverflow.com/questions/2242715/methods-of-metaclasses-on-class-instances) — Mark, May 26 '22 at 23:32
"I thought that the class attribute defined in the metaclass would be accessible from self.myattribute in the instances" no, instances look up attributes in the *class namespaces*, not the *metaclass* namespaces. Note, classes *are instances of metaclasses*, so *they* look up attributes in their (meta)class namespace — juanpa.arrivillaga, May 26 '22 at 23:35
@juanpa.arrivillaga ah I see. But then can I access that namespace from an instance? Ideally without necessarily having to to import Foo everywhere I might need to do a lookup? Or is using a metaclass a completely wrong approach? Although it seems fitting to me. — logicOnAbstractions, May 27 '22 at 00:09
FWIW, this use case may be better served by the ``__init_subclass__`` method. That may let you avoid writing a metaclass. — Raymond Hettinger, May 28 '22 at 01:38

score 2 · Accepted Answer · answered May 27 '22 at 01:13

2

Meta describes how to create class but not what class that will be.

Meta != Parent with inherited attributes

So you have to pass proper attributes into new class:

class Meta(type):
    _a = {}
    def __new__(meta, name, bases, attrs):
        if "d" in attrs:
            meta._a.update(attrs["d"])
        attrs["a"] = meta._a
        return type.__new__(meta, name, bases, attrs)

class Data:
    pass

class DataTable(Data, metaclass=Meta):
    pass

class Foo(DataTable):
    d = {"a":1}

class Bar(DataTable):
    d = {"b":2}

f = Foo()
print(Foo.a)
print(f.a)

{'a': 1, 'b': 2}
{'a': 1, 'b': 2}

answered May 27 '22 at 01:13

rzlvmp

7,512
5
16
45

Armph. Yeah that makes a lots of sense actually. Thanks a bunch. – logicOnAbstractions May 27 '22 at 01:21
Just to reformulate for myself (and others who might read this). The gist of it is that meta.foo is a metaclass's attribute. attrs (the last arg in __new__) contains/will contain the class that is currently being created's attributes. Hence the need to pop the ALLDEPENDANCIES in there. – logicOnAbstractions May 27 '22 at 12:05

score 1 · Answer 2 · answered May 28 '22 at 01:14

Instance class attribute search does not go into the metaclass - just to the class. The metaclass could set ALLDEPENDANCIES in each new class, with a single line in its __new__, but if you want cleaner code, in the sense the dictionary is not aliased everywhere, you can just access the attribute through the class.

Using your code, as is:

Foo().__class__.ALLDEPENDANCIES

will work from anywhere (just as `type(Foo()).ALLDEPENDANCIES).

In order to set the attribute in the new classes, so that it will be visible in the newly created classes, an option is:


from types import MappingProxyType

class Meta(type):
    ALLDEPENDANCIES = {}
    ALLDEPSVIEW = MappingProxyType(ALLDEPENDANCIES)
    def __new__(meta, name, bases, attrs):
        if "DEPENDANCIES" in attrs.keys():
            for key, value in attrs.items():
                if key == "DEPENDANCIES":
                    meta.ALLDEPENDANCIES.update(attrs["DEPENDANCIES"])
        new_cls = super().__new__(meta, name, bases, attrs)
        new_cls.ALLDEPENDANCIES = meta.ALLDEPSVIEW
        return new_cls

(Inserting the new attr in attrs before calling type.__new__ will also work)

Here I do two other extras: (1) call super().__new__ instead of hardcoding a call to type.__new__: this allows your metaclass to be composable with other metaclasses, which might be needed if one of your classes will cross with other metaclass (for example, if you are using abstract base classes from abc or collections.abc). And (2) using a MappingProxyType which is a "read only" dictionary view, and will stop acidental direct updates of the dict through classes or instances.

Python - using metaclass & class attributes in object instances

2 Answers2