Making copies of built-ins classes

Question

I'm trying to write function which creates classes from classes without modifying original one.

Simple solution (based on this answer)

def class_operator(cls):
    namespace = dict(vars(cls))
    ...  # modifying namespace
    return type(cls.__qualname__, cls.__bases__, namespace)

works fine except type itself:

>>> class_operator(type)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: type __qualname__ must be a str, not getset_descriptor

Tested on Python 3.2-Python 3.6.

(I know that in current version modification of mutable attributes in namespace object will change original class, but it is not the case)

Update

Even if we remove __qualname__ parameter from namespace if there is any

def class_operator(cls):
    namespace = dict(vars(cls))
    namespace.pop('__qualname__', None)
    return type(cls.__qualname__, cls.__bases__, namespace)

resulting object doesn't behave like original type

>>> type_copy = class_operator(type)
>>> type_copy is type
False
>>> type_copy('')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object
>>> type_copy('empty', (), {})
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object

Why?

Can someone explain what mechanism in Python internals prevents copying type class (and many other built-in classes).

Answer 1

The problem here is that type has a __qualname__ in its __dict__, which is a property (i.e. a descriptor) rather than a string:

>>> type.__qualname__
'type'
>>> vars(type)['__qualname__']
<attribute '__qualname__' of 'type' objects>

And trying to assign a non-string to the __qualname__ of a class throws an exception:

>>> class C: pass
...
>>> C.__qualname__ = 'Foo'  # works
>>> C.__qualname__ = 3  # doesn't work
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only assign string to C.__qualname__, not 'int'

This is why it's necessary to remove the __qualname__ from the __dict__.

As for the reason why your type_copy isn't callable: This is because type.__call__ rejects anything that isn't a subclass of type. This is true for both the 3-argument form:

>>> type.__call__(type, 'x', (), {})
<class '__main__.x'>
>>> type.__call__(type_copy, 'x', (), {})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object

As well as the single-argument form, which actually only works with type as its first argument:

>>> type.__call__(type, 3)
<class 'int'>
>>> type.__call__(type_copy, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: type.__new__() takes exactly 3 arguments (1 given)

This isn't easy to circumvent. Fixing the 3-argument form is simple enough: We make the copy an empty subclass of type.

>>> type_copy = type('type_copy', (type,), {})
>>> type_copy('MyClass', (), {})
<class '__main__.MyClass'>

But the single-argument form of type is much peskier, since it only works if the first argument is type. We can implement a custom __call__ method, but that method must be written in the metaclass, which means type(type_copy) will be different from type(type).

>>> class TypeCopyMeta(type):
...     def __call__(self, *args):
...         if len(args) == 1:
...             return type(*args)
...         return super().__call__(*args)
... 
>>> type_copy = TypeCopyMeta('type_copy', (type,), {})
>>> type_copy(3)  # works
<class 'int'>
>>> type_copy('MyClass', (), {})  # also works
<class '__main__.MyClass'>
>>> type(type), type(type_copy)  # but they're not identical
(<class 'type'>, <class '__main__.TypeCopyMeta'>)

---

There are two reasons why type is so difficult to copy:

1. It's implemented in C. You'll run into similar problems if you try to copy other builtin types like int or str.

2. The fact that type is an instance of itself:

   >>> type(type)
   <class 'type'>
   

   This is something that's usually not possible. It blurs the line between class and instance. It's a chaotic accumulation of instance and class attributes. This is why __qualname__ is a string when accessed as type.__qualname__ but a descriptor when accessed as vars(type)['__qualname__'].

---

As you can see, it's not possible to make a perfect copy of type. Each implementation has different tradeoffs.

The easy solution is to make a subclass of type, which doesn't support the single-argument type(some_object) call:

import builtins

def copy_class(cls):
    # if it's a builtin class, copy it by subclassing
    if getattr(builtins, cls.__name__, None) is cls:
        namespace = {}
        bases = (cls,)
    else:
        namespace = dict(vars(cls))
        bases = cls.__bases__

    cls_copy = type(cls.__name__, bases, namespace)
    cls_copy.__qualname__ = cls.__qualname__
    return cls_copy

The elaborate solution is to make a custom metaclass:

import builtins

def copy_class(cls):
    if cls is type:
        namespace = {}
        bases = (cls,)

        class metaclass(type):
            def __call__(self, *args):
                if len(args) == 1:
                    return type(*args)
                return super().__call__(*args)

        metaclass.__name__ = type.__name__
        metaclass.__qualname__ = type.__qualname__
    # if it's a builtin class, copy it by subclassing
    elif getattr(builtins, cls.__name__, None) is cls:
        namespace = {}
        bases = (cls,)
        metaclass = type
    else:
        namespace = dict(vars(cls))
        bases = cls.__bases__
        metaclass = type

    cls_copy = metaclass(cls.__name__, bases, namespace)
    cls_copy.__qualname__ = cls.__qualname__
    return cls_copy