What is the difference between type and type.__new__ in python?

Question

I was writing a metaclass and accidentally did it like this:

class MetaCls(type):
    def __new__(cls, name, bases, dict):
        return type(name, bases, dict)

...instead of like this:

class MetaCls(type):
    def __new__(cls, name, bases, dict):
        return type.__new__(cls, name, bases, dict)

What exactly is the difference between these two metaclasses? And more specifically, what caused the first one to not work properly (some classes weren't called into by the metaclass)?

Answer 1

In the first example you're creating a whole new class:

>>> class MetaA(type):
...     def __new__(cls, name, bases, dct):
...         print 'MetaA.__new__'
...         return type(name, bases, dct)
...     def __init__(cls, name, bases, dct):
...         print 'MetaA.__init__'
... 
>>> class A(object):
...     __metaclass__ = MetaA
... 
MetaA.__new__
>>> 

while in the second case you're calling parent's __new__:

>>> class MetaA(type):
...     def __new__(cls, name, bases, dct):
...         print 'MetaA.__new__'
...         return type.__new__(cls, name, bases, dct)
...     def __init__(cls, name, bases, dct):
...         print 'MetaA.__init__'
... 
>>> class A(object):
...     __metaclass__ = MetaA
... 
MetaA.__new__
MetaA.__init__
>>> 

Answer 2

The first thing you need to figure out is how object.__new__() works.

Here it is from the documentation below:

object.__new__(cls[, ...])

Called to create a new instance of class cls. __new__() is a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument. The remaining arguments are those passed to the object constructor expression (the call to the class). The return value of __new__() should be the new object instance (usually an instance of cls).

Typical implementations create a new instance of the class by invoking the superclass’s __new__() method using super(currentclass, cls).__new__(cls[, ...]) with appropriate arguments and then modifying the newly-created instance as necessary before returning it.

If __new__() returns an instance of cls, then the new instance’s __init__() method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to __new__().

If __new__() does not return an instance of cls, then the new instance’s __init__() method will not be invoked.

__new__() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.

So in mg.'s answer, the former doesn't call function __init__ while the latter calls function __init__ after calling __new__.

Answer 3

Please refer to the annotation below, hope this helpful.

class MetaCls(type):
    def __new__(cls, name, bases, dict):
        # return a new type named "name",this type has nothing
        # to do with MetaCls,and MetaCl.__init__ won't be invoked
        return type(name, bases, dict)

class MetaCls(type):
    def __new__(cls, name, bases, dict):
        # return a new type named "name",the returned type 
        # is an instance of cls,and cls here is "MetaCls", so 
        # the next step can invoke MetaCls.__init__ 
        return type.__new__(cls, name, bases, dict)

Answer 4

class MyMeta(type):
    def __new__(meta, cls, bases, attributes):
        print 'MyMeta.__new__'
        return type.__new__(meta, cls, bases, attributes)
    def __init__(clsobj, cls, bases, attributes):
        print 'MyMeta.__init__'

class MyClass(object):
  __metaclass__ = MyMeta
  foo = 'bar'

Another way to achieve the same result:

cls = "MyClass"
bases = ()
attributes = {'foo': 'bar'}
MyClass = MyMeta(cls, bases, attributes)

MyMeta is a callable so Python will use the special method __call__.

Python will look for __call__ in the MyMeta's type (which is type in our case)

"For new-style classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary"

MyClass = MyMeta(...) is interpreted as:

my_meta_type = type(MyMeta)
MyClass = my_meta_type.__call__(MyMeta, cls, bases, attributes)

Inside the type.__call__() I imagine something like this:

MyClass = MyMeta.__new__(MyMeta, cls, bases, attributes)
meta_class = MyClass.__metaclass__
meta_class.__init__(MyClass, cls, bases, attributes)
return MyClass

MyMeta.__new__() will decide how the MyClass is built:

type.__new__(meta, cls, bases, attributes) will set the correct metaclass (which is MyMeta) for MyClass

type(cls, bases, attributes) will set the default metaclass (which is type) for MyClass

Answer 5

return type(name, bases, dict)

What you get back from this is a new type, and not a MetaCls instance at all. Consequently, your methods defined in MetaCls (including __init__) can't ever be called.

type.__new__ will be called as part of creating that new type, yes, but the value of cls going into that function is going to be type and not MetaCls.

Answer 6

It's all described pretty well here.

If you don't return the right type of object, there's no point to defining a custom metaclass.