Python: How to annotate a variable number of iterable attributes?

Question

I have a class family for which I need to be able to iterate through attributes of type: Metric.

The family consists of an abstract base class parent and child classes. The child classes will all have varying number of class attributes of type Metric, and they inherit an __iter__ method from the parent class that allows me to iterate through the attributes.

I am using iterable attributes rather than a dict because I want my objects to be typed, but I need to be able to call metrics in sequence, and by name. So I need to be able to do:

Metrics.metric_1 and for metric in Metrics:

My question is, how do I correctly hint in the base class that there are a variable number of attributes of the same type?

I'm currently using a couple of attribute hints with an ellipsis:

class MetricsBase(ABC):
    metric_1: Metric
    metric_2: Metric
    ...

    @classmethod
    def __iter__(cls):
        for attr, value in cls.__dict__.items():
            if not attr.startswith("__"):
                yield value

class MetricChild(MetricsBase):
    metric_1 = Metric(x)
    metric_2 = Metric(y)
    metric_3 = Metric(z)

But I'm not sure if this is pythonic or correct, and wondering if there is a neater way of doing this.

Many thanks for any input!

jsbueno · Accepted Answer · 2023-02-15T14:34:43.020

I am not answering how to "fix" static type checking on that.

That said, this is ok as Python code, hence "pythonic" . the problem is that you want to use static type checking on it - and you are using a dynamic meta programming technique there. Static type checking is not meant to check this (in a way of saying, it can only handle a small subset of what would be "pythonic"). Maybe there is a way to "solve" this - but if you can, just mark the static type checkers to skip that, and spare hours yourself hours of meaningless work (since it won't change how the code works)

More important than that, that __iter__ method won't work for the class itself, regardless of you marking it as a @classmethod. (It will work fot the instances, despite you doing so, though). If you want to iterate on the class, you will have to resort to a metaclass:

import abc

class MetricMeta(abc.ABCMeta):
    def __iter__(cls):
        # this will make _instances of this metaclass__ iterable
        for attr, value in cls.__dict__.items():
            if not attr.startswith("__"):
                yield value


class MetricsBase(metaclass=MetricsMeta):
    metric_1: Metric
    metric_2: Metric
    ...

Type chekers actually, should, supposedly, not need one to expliclty annotate all variables, reducing Python to a subset of Pascal or the like. If you simply type in your class attributes in each subclass, attributing a Metric instance to then it should work, without the need to explictly annotate each one with a :Metric.
They will certainly complain when you try to iterate over a class with a statement like for metric in Metrics:, but that is easily resolvable by asserting to it explicitly that the class is iterable, using typing.cast. No tool (at least not yet) will be able to "see" that the metaclass you are using feature an __iter__ method that enables the class itself to be iterable.

from typing import cast
from collections.abc import Iterable

...

for metric in cast(Iterable, metrics):
    ...

Python: How to annotate a variable number of iterable attributes?

1 Answers1