Trying to understand dict for Python Classes vs Instances

Question

I understand that you can view things within Python as attributes as per this article. But I am curious as to why in an instance of a class, parent_instance, we do not have the same attributes stored within the ParentClass. This question said that, "The '__dict__' and '__weakref__' entries in a class's __dict__ (when present) are descriptors used for retrieving an instance's dict pointer and weakref pointer"

I know that with a dir(parent_instace) call, I can still obtain a '__weakref__', which will be None, and the method_1 and method_2. But why aren't these within __dict__ in a similar format only set to None?

What is setting or defining what is in __dict__? Why can I not access method attributes within an instance using __dict__. I understand it's not within the scope, but why is it not in the scope? Shouldn't the method attributes be within the instance since it can access them?

Reference Code:

class ParentClass:
    def __init__(self):
        self.int_1: int = 0
        self.str_1: str = ''

    def method_1(self):
        pass

    def method_2(self):
        pass


parent_instance = ParentClass()

print("Parent Class Attributes:")
for atr in ParentClass.__dict__:
    print(atr)

print("\nParent Instance Attributes:")
for atr in parent_instance.__dict__:
    print(atr)

You don't have methods in instance `__dict__` to avoid extreme memory overhead: say you have a large class (50 methods - not too much) and 1000 copies of it. If they have a copy of all methods (even references), you have just wasted approx. 200 KB of memory. (50 and 1000 are really small numbers, you may have 10^5 class instances easily) — STerliakov, Jul 26 '22 at 16:51
That makes sense, until you start using getattr and setattr to manipulate the methods in a particular instance. I suppose that then those would have to be tracked, but the 'known' methods could just be resolved back to the Parent Class. — Lucian, Jul 26 '22 at 16:55
When you alter an instance method (if it is possible, of course), it is added to instance `__dict__`. And instance `__dict__` has higher lookup precedence. (and `getattr` is no different from regular dot access in terms of value resolution) — STerliakov, Jul 26 '22 at 17:04
"But I am curious as to why in an instance of a class, parent_instance, we do not have the same attributes stored within the ParentClass" **because that is the whole point**. `__dict__` is the **namespace of the obejct**. Classes and their instance have *seperate namespaces*. — juanpa.arrivillaga, Jul 26 '22 at 17:34
"What is setting or defining what is in `__dict__`?" **whatever assigns to that object's namespace**. The class is a seperate object from an instance of that class. — juanpa.arrivillaga, Jul 26 '22 at 17:35
@Lucian you cannot manipulate methods on a particular instance, methods (that is, functions that will use their descriptor protocol to dynamically bind instances as the first argument) belong to the *class*. When you assign to an instance, `my_instance.some_method = whatever`, you are *shadowing* the method on the instance (with some other callable) — juanpa.arrivillaga, Jul 26 '22 at 17:38
By manipulate methods, I simply meant that we can add methods using setattr() and set an attribute to another defined method. But, that creates more questions about the difference between methods from the original class and 'pseudo'-methods added to the instance using setattr(). Because, you are just setting some attribute to some method. I do not know if they are seen the same inside the object. — Lucian, Jul 26 '22 at 18:02
I was not correlating namespace with `__dict__` previous, that clarifies that. I wasn't considering the concept of *(local, global, built-in namespace)* but that would explain a few things. https://mail.python.org/pipermail/tutor/2008-January/059823.html — Lucian, Jul 26 '22 at 18:07
@Lucian they aren't, I'm trying to explain to you that you are *shadowing* the method on the class with an attribute on the instance. The very same as if you have a class, `class Foo:` with a single `bar = 0`, if you then do `foo = Foo(); foo.bar = 88; print(foo.bar, Foo.bar)` — juanpa.arrivillaga, Jul 26 '22 at 18:21
Would you say it is bad practice to shadow the method on the class with an attribute on the instance? @juanpa.arrivillaga — Lucian, Jul 26 '22 at 22:35

Raymond Hettinger · Answer 1 · 2022-07-26T18:18:59.777

Question 1: The separation of instance data and class data

You asked a lot of questions, but I think the common theme is "why doen't the instance dictionary store everything that is accessible from an instance?"

The answer is that instances store information unique to each instance while classes store information (methods and class variables) that are shared by all instances.

This is central to the notion of what it means to be class (what all instances have in common), and it is memory efficient (one class and many instances).

Users don't normally access the class __dict__ or instance __dict__ directly. They use dir() to see all the levels collapsed to together and use either the dot operator or getattr() to do a look up in the chain of dictionaries (instance first and the MRO of class dictionaries). From the user's point of view, almost everything is in the object.

There are some implementation nuances such as __mro__ being only accessible from the class and not from the instance. Also, the garbage collection, refcount, and weak reference support aren't shown in the instance dictionary even though they are part of the object.

Question 2: Showing all attributes at the class level

A secondary question is "why don't classes list all attributes including instance attributes?"

There are two equivalent answers:

a) Since instance dictionaries are dynamically updatable and store both the key and value, there is no need to store this information at the class level.

b) Since instance attributes can be added after instantiation, the class cannot know in advance which attributes are going to be stored in instances.

This is true most of the time in Python (and other interpreted OOP languages). However, classes defined with __slots__ are an exception to this rule because they don't use instance dictionaries. Dataclasses may seem like an exception because they store code generation metadata at the class level, but after the code is generated they are just regular classes and the same principles apply.

Trying to understand __dict__ for Python Classes vs Instances

1 Answers1

Question 1: The separation of instance data and class data

Question 2: Showing all attributes at the class level

Trying to understand dict for Python Classes vs Instances