Python: Filter iterable class

Question

Is there a hook/dunder that an Iterable object can hold so that the builtin filter function can be extended to Iterable classes (not just instances)?

Of course, one can write a custom filter_iter function, such as:

def filter_iter(filt_func: callable, collection_cls: type):
    name = 'Filtered' + collection_cls.__name__  # would this automatic scheme lead to namespace conflicts?
    wrapped_cls = type(name, (collection_cls,), {'_filt_func': staticmethod(filt_func)})
    def __iter__(self):
        yield from filter(self._filt_func, super(wrapped_cls, self).__iter__())
    wrapped_cls.__iter__ = __iter__
    return wrapped_cls

which would have the desired effect. For example,

from collections import Collection, Iterable
class Chunker(Iterable):
    def __init__(self, source: Iterable, chk_size: int=2):
        self._source = source
        self._chk_size = chk_size
    def __iter__(self):
        yield from zip(*([iter(self._source)] * self._chk_size))


chunker = Chunker(range(12), 2)
assert list(chunker) == [(0, 1), (2, 3), (4, 5), (6, 7), (8, 9), (10, 11)]
FilteredChunker = filter_iter(lambda x: sum(x) % 3 == 0, Chunker)
filtered_chunker = FilteredChunker(range(12))
assert list(filtered_chunker) == [(4, 5), (10, 11)]

But, just as there's an __iter__ hook that determines how to iterate over an object (for example, how list should behave when called on the object), is there a sort of __filter__ hook to determine how filter should behave when called on that object?

If not, what are the best practices or standards around filtering iterables?

I would expect filter to call iter in order to access each item. And then filter will do it's magic. — Torxed, May 25 '20 at 13:12
No, because an instance of `filter` (yes, `filter` is a class, not a function) is iterable by iterating over the original iterable argument and being selective about which items it actually yields. Why are you trying to filter `Chunker` (which is *not* iterable; *instances* of `Chunker` are iterable) in the first place? — chepner, May 25 '20 at 13:18
Hi again @chepner! Yes, sorry, I should have said callable (and I did discover this, since I actually did `filter.mro()` to see if there were any goodies there I could use to hack). As to the "why not just use `filter` on instances"; I think that same question could be asked, and answered, in general. One such general answer is: Because you want a class whose instances have a specific behavior: Be it caching, filtering, or what ever. — thorwhalen, May 25 '20 at 13:56
Ah, I see what you are getting at. I though you were trying to iterate over the class, not redefine the existing `__iter__` for the class. See my answer. — chepner, May 25 '20 at 14:07

score 0 · Accepted Answer · edited May 26 '20 at 15:26

0

Unlike with list (and __iter__ for instance), there is no such hook for filter. The latter is just an application of the iterator protocol, not a separate protocol in and of itself.

To not leave you empty handed, here is a more concise version of the filtered_iter you proposed, that dynamically subclasses the given class, composing its __iter__ method with filter.

def filter_iter(p, cls):
    class _(cls):
        def __iter__(self):
            yield from filter(p, super().__iter__())
    return _

edited May 26 '20 at 15:26

thorwhalen

1,920
14
26

answered May 25 '20 at 14:06

chepner

497,756
71
530
681

Isn't your proposed solution just a more elegant version of the `filter_iter` I included in my question already as what I **could** do, but would prefer to use the builtin `filter` function itself. Such a solution would probably require defining its behavior when applied to a class by some kind of magic method (such as how one defines the behavior of `iter` and `list` through `__iter__`, or `len` through `__len__`). – thorwhalen May 25 '20 at 19:06
Another possible solution route would be to create a diamond mro to be able to intercept the `__init__` of the builtin `filter` and inject the desired behavior for classes. Something like `class filter(Filtered, builtins.filter): ...`. But it starts to approach dark arts I'm not used to. – thorwhalen May 25 '20 at 19:27
There is no such hook. `filter` is just an application of the iterator protocol, not a separate protocol in and of itself. – chepner May 25 '20 at 20:37
Not sure what the right way to do this is -- but I'll just edit your answer to make it more aligned with my question, and accept it as the answer. – thorwhalen May 26 '20 at 14:16

Python: Filter iterable class

1 Answers1