Python 2.6, 3 abstract base class misunderstanding

Question

I'm not seeing what I expect when I use ABCMeta and abstractmethod.

This works fine in python3:

from abc import ABCMeta, abstractmethod

class Super(metaclass=ABCMeta):
    @abstractmethod
    def method(self):
        pass

a = Super()
TypeError: Can't instantiate abstract class Super ...

And in 2.6:

class Super():
    __metaclass__ = ABCMeta
    @abstractmethod
    def method(self):
         pass

a = Super()
TypeError: Can't instantiate abstract class Super ...

They both also work fine (I get the expected exception) if I derive Super from object, in addition to ABCMeta.

They both "fail" (no exception raised) if I derive Super from list.

I want an abstract base class to be a list but abstract, and concrete in sub classes.

Am I doing it wrong, or should I not want this in python?

Answer 1

With Super build as in your working snippets, what you're calling when you do Super() is:

>>> Super.__init__
<slot wrapper '__init__' of 'object' objects>

If Super inherits from list, call it Superlist:

>>> Superlist.__init__
<slot wrapper '__init__' of 'list' objects>

Now, abstract base classes are meant to be usable as mixin classes, to be multiply inherited from (to gain the "Template Method" design pattern features that an ABC may offer) together with a concrete class, without making the resulting descendant abstract. So consider:

>>> class Listsuper(Super, list): pass
... 
>>> Listsuper.__init__
<slot wrapper '__init__' of 'list' objects>

See the problem? By the rules of multiple inheritance calling Listsuper() (which is not allowed to fail just because there's a dangling abstract method) runs the same code as calling Superlist() (which you'd like to fail). That code, in practice (list.__init__), does not object to dangling abstract methods -- only object.__init__ does. And fixing that would probably break code that relies on the current behavior.

The suggested workaround is: if you want an abstract base class, all its bases must be abstract. So, instead of having concrete list among your bases, use as a base collections.MutableSequence, add an __init__ that makes a ._list attribute, and implement MutableSequence's abstract methods by direct delegation to self._list. Not perfect, but not all that painful either.

Answer 2

Actually, the issue is with __new__, rather than with __init__. Example:

from abc import ABCMeta, abstractmethod
from collections import OrderedDict

class Foo(metaclass=ABCMeta):
    @abstractmethod
    def foo(self):
        return 42

class Empty:
    def __init__(self):
        pass

class C1(Empty, Foo): pass
class C2(OrderedDict, Foo): pass

C1() fails with a TypeError as expected, while C2.foo() returns 42.

>>> C1.__init__
<function Empty.__init__ at 0x7fa9a6c01400>

As you can see, it's not using object.__init__ nor is it even invoking its superclass (object) __init__

You can verify it by calling __new__ yourself:

C2.__new__(C2) works just fine, while you'll get the usual TypeError with C1.__new__(C1)

So, imho it's not as clear cut as

if you want an abstract base class, all its bases must be abstract.

While that's a good suggestion, the converse it's not necessarily true: neither OrderedDict nor Empty are abstract, and yet the former's subclass is "concrete", while the latter is "abstract"

If you're wondering, I used OrderedDict in the example instead of list because the latter is a "built-in" type, and thus you cannot do:

OrderedDict.bar = lambda self: 42

And I wanted to make it explicit that the issue is not related to it.