response of `vars(self)` is missing attributes

Question

I have defined a class and need most of its attributes handed over to a function for processing. So I thought instead of creating a huge mess and naming them all I'd do processSomething(vars(self)) and hand over a nice dictionary with all attributes and values.

But I found that almost all of the attributes are missing.

I have halted the code in the debugger and ran a few tests:

>>> vars(self).keys()
dict_keys(['date_expire', 'tokenUrl', 'settings', 'requestSession', 'ttlDays'])

These are 5. While I am expecting about 20 attributes, as per __slots__ definition of my class here:

__slots__ = (
    'token',        # the invitation-token to load the username with
    'request',      # the http-request which is currently being serviced
    'inhibit',      
     #... many lines deleted
    'templateObj',  # an instance of myMail.template.Template
    'emailBody',    # what will be sent via SMTP
    'warningsAr',   # messages for the client
)

I can see the attributes in the debugger window and can access them directly. I have read the manual of vars() and could not find any switches. Interestingly, dir(self) shows all attribute-names, but no values. So I cannot use that. But I thought vars and dir should show the same?

I guess I'll build a workaround, but I really want to understand what is happening here. Can you please help?

Answer 1

vars() returns you the __dict__ namespace of an instance. But __slots__ attributes are not stored in the __dict__ namespace. That's their whole point.

Instead, Python creates dedicated pointer slots in the instance memory structure for each value, and uses descriptors on the class object to retrieve those values. So inst.attr is translated to type(inst).attr.__get__(inst) to return that value.

From the __slots__ documentation:

The __slots__ declaration takes a sequence of instance variables and reserves just enough space in each instance to hold a value for each variable. Space is saved because __dict__ is not created for each instance.

[...]

• __slots__ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name.

Note that having a __dict__ anyway is usually a sign you forgot to make a subclass use __slots__ for their attributes, or you inherited from a base Python class that didn't use __slots__ itself. Proper use of __slots__ should result in instances with no __dict__ namespace; their goal is to reduce memory by avoiding the sparse hash table required for a dict (which can waste space).

From the documentation again:

• When inheriting from a class without __slots__, the __dict__ attribute of that class will always be accessible, so a __slots__ definition in the subclass is meaningless.

[...]

• The action of a __slots__ declaration is limited to the class where it is defined. As a result, subclasses will have a __dict__ unless they also define __slots__ (which must only contain names of any additional slots).

If you wanted to list all available instance fields, you'd have to include the __slots__ enumeration from the class, and not just look at vars():

from itertools import chain

def slots_for_instance(inst):
    def _slots_for_class(c):
        slots = getattr(c, '__slots__', ())
        if isinstance(slots, str):
            # __slots__ can be a string naming a single attribute
            slots = (slots,)
    return set(chain.from_iterable(
        getattr(_slots_for_class(c) for c in type(inst).__mro__))