0

I have a behaviour that I cannot explain in my code: I want to implement a dictionary class that adds the following two behaviours to standard dictionaries:

  • It has some associated object and whenever a key is not available in the dictionary it tries to generate a value to that key by calling a method from the associated object
  • It can have a maximum length and whenever we try to add an item that would increase the size of the dictionary beyond its maximum length the dictionary is reduced in a FIFO manner by forgetting the oldest (key,item) pair

I have somewhat achieved this behaviour by subclassing from collections.OrderedDict (to have a notion of oldest (key,item) pair) and overwriting the __setitem__ (to enforce maximum len) and __missing__ (to generate missing keys on the fly) methods:

from collections import OrderedDict
from weakref import proxy

class AttachedAutoLimitedDict(OrderedDict):
    """A special dictionary implementation that is meant to be used together with an associated object.

    When a not-existing key is requested from the dictionary it attempts to generate a value to that key by calling
    the generation method of the associated object. Furthermore the dictionary can be limited in size 
    by setting the max_len property during construction."""

    def __init__(self, associated_object, max_len: int | None = None, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.associated_object = proxy(associated_object)
        if max_len is None:
            self.__setitem__ = super().__setitem__
        else:
            self.__setitem__ = self.__setitemmaxlen__
        self.max_len = max_len

    def __missing__(self, key):
        """Overwrites the __missing__ implementation. Attempts to generate a value for a missing key
        by using the key as argument to the generation method of the associated object."""
        try:
            self.__setitem__(key, self.associated_object.generation_method(key))
            return self[key]
        except AttributeError:
            super().__missing__(key)

    def __setitemmaxlen__(self, key, value):
        """This ensures that the dictionary does not grow beyond max_len"""
        # have to do it with the +1 since len(self) is incremented before the __setitem__ call
        if len(self) >= self.max_len + 1:
            # this is inherited from OrderedDict, last=False specifies FIFO popping
            self.popitem(last=False)
        super().__setitem__(key, value)

Now the implementation seems to be working like this (haven't fully tested it yet). But I had a problemt when in the implementation of __missing__ I used self[key]=self.associated_object.generation_method(key) instead of self.__setitem__(key, self.associated_object.generation_method(key)), because the custom implementation of __setitem__ was never called (and hence the restriction on len was not enforced).
Does somebody know why that happens?

chriss
  • 53
  • 7
  • 1
    not an answer, but the first half of the if clause seems unnecessary ... `self.__setitem__ = super().__setitem__` is already the case if you do nothing – Anentropic Aug 17 '23 at 12:31
  • 1
    [This](https://stackoverflow.com/questions/2129643/dynamically-assign-special-methods-to-objects-but-not-classes-in-python) would explain your problem. In short, you cannot assign to special methods. – ken Aug 17 '23 at 13:18
  • Thanks @ken, so this was due to my silly optimization of trying to replace dynamically replace __setitem__ in the instance initialization only if max_len is not None to avoid checking self.max_len for None in every call to __setitem__ – chriss Aug 17 '23 at 14:13

1 Answers1

0

As per @ken's suggestion in the comment, python looks up dunder methods on the class, not on the instance. Therefore dynamically assigning to __setitem__ in the instance initialization has no effect. The below works:

class AttachedAutoLimitedDict(OrderedDict):
    """A special dictionary implementation that is meant to be used together with an associated object.

    When a not-existing key is requested from the dictionary it attempts to generate a value to that key by calling
    the generation method of the associated object. Furthermore the dictionary can be limited in size 
    by setting the max_len property during construction."""

    def __init__(self, associated_object, max_len: int | None = None, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.associated_object = proxy(associated_object)
        self.max_len = max_len

    def __missing__(self, key):
        """Overwrites the __missing__ implementation. Attempts to generate a value for a missing key
        by using the key as argument to the generation method of the associated object."""
        try:
            self[key]=self.associated_icon_object.get_run_data(key)
            return self[key]
        except AttributeError:
            super().__missing__(key)

    def __setitem__(self, key, value):
        """This ensures that the dictionary does not grow beyond max_len"""
        if self.max_len is not None and len(self) >= self.max_len + 1:
            self.popitem(last=False)
        super().__setitem__(key, value)
chriss
  • 53
  • 7