What is the downside of using object.new in new?

Question

Coding exception classes, I came across this error:

TypeError: object.__new__(A) is not safe, use Exception.__new__()

There's a similar question posted here: TypeError: object.__new__(int) is not safe, use int.__new__(). So __new__ was deprecated for the following reason:

[Python-Dev] __new__ deprecation

Guido van Rossum

"The message means just what it says. :-) There's no point in calling object.__new__() with more than a class parameter, and any code that did so was just dumping those args into a black hole."

But the warning in 3.3 that I get "is not safe" is scary. I try to understand the implication of using object.__new__, let's consider this example:

>>> class A(Exception):
...     def __new__(cls, *args):
...             return object.__new__(A)
...
>>> A()
TypeError: object.__new__(A) is not safe, use Exception.__new__()

Fails miserably. Another Example:

>>> class A(object):
...     def __new__(cls, *args):
...             return object.__new__(A)
...
>>>
>>> A()
<__main__.A object at 0x0000000002F2E278>

works fine. Although, object is a builtin class just like Exception with respect to their roles, they share the trait of being builtin-classes. Now with Exception, the first example raises TypeError, but with object, it does not?

(a) What are the downsides of using object.__new__ that made Python to raise the error (TypeError:...is not safe...) in the first Example?

(b) What sort of checking Python performs before to calling __new__? Or: What is the condition that makes Python raise the error in the first example?

score 9 · Accepted Answer · answered Sep 15 '16 at 19:35

There is no problem in calling object.__new__, but there is a problem in not calling Exception.__new__.

Exception class was designed in such way that it is crucial that its __new__ must be called, so it complains if that is not done.

There was a question why this happens only with built-in classes. Python in fact does it with every class which is programmed to do that.

Here is a simplified poor-mans implementation of the same mechanism in a custom class:

class A(object):
    def __new__(cls):
        rtn = object.__new__(cls)
        rtn.new_called = True
        return rtn

    def __init__(self):
        assert getattr(self,'new_called',False), \
            "object.__new__ is unsafe, use A.__new__"

class B(A):
    def __new__(cls):
        return object.__new__(cls)

And now:

>>> A()
<__main__.A object at 0x00000000025CFF98>

>>> B()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in __init__
AssertionError: object.__new__ is unsafe, use A.__new__

As a side note, this example from the question actually has two errors:

>>> class A(Exception):
...     def __new__(cls, *args):
...             return object.__new__(A)

The first is that __new__ is called on object, thus ignoring Exception.__new__.

The other, just as severe is that A is passed to __new__ instead of cls, which hinders all classes inherited from A.

See this example:

class A(object):
    def __new__(cls):
        return object.__new__(A)  # The same erroneous line, without Exception

class B(A):
    pass

And now B() does not create an instance of B:

>>> B()
<__main__.A object at 0x00000000025D30B8>

I just realized the mechanism is coded in `object.__new__`. Look at [this code](https://gist.github.com/anonymous/9a98b59f1f3bc518f0ba89ff667083c6). That is, `object.__new__` raises the `TypeError`. Very reasonable now to say "not safe." Although, overriding `Exception.__new__` in a subclass won't raise an error, even if `Exception.__new__` isn't called inside `Subclass.__new__`, this is equivalently dangerous. — GIZ, Sep 16 '16 at 06:04

score 0 · Answer 2 · edited Sep 15 '16 at 16:12

0

Calling object.__new__(A) returns an instance of A, but does so without calling Exception.__new__() if it is defined.

edited Sep 15 '16 at 16:12

wim

338,267
99
616
750

answered Sep 15 '16 at 16:04

chepner

497,756
71
530
681

_"but does so without calling `Exception.__new__()` if it is defined._ This wouldn't be a problem because we defined `__new__` lower (in the subclass). This has nothing to do with whether Exception.__new__ will be called or not. If we have two superclasses each having `__new__` method : `C(A,B)`, then the left-most `__new__` will be called, that is `A.__new__` and `B.__new__` will not be called. Could you elaborate with an example? – GIZ Sep 15 '16 at 16:53
What you raise is a separate problem, solved by either calling both `A.__new__()` and `B.__new__()` explicitly, or by using `super` and ensuring all the ancestor classes use it, too. Failing to call `Exception.__new__` is a big problem, if you miss `Exception`-specific configuration and plan on passing your object to someone expecting a properly configured instance of `Exception`. – chepner Sep 15 '16 at 17:32
I do see your point: (replacing `__new__` with another `__new__` may skip some or all the crucial configurations required for an object during creation). – GIZ Sep 15 '16 at 18:44
If you consider [this code](https://gist.github.com/anonymous/49e7ccb7e83e02c773d64b8913d95eff) you will notice that Python raises (`TypeError:... is not safe...`) only for builtin classes. You can see from [the code](https://gist.github.com/anonymous/49e7ccb7e83e02c773d64b8913d95eff) that class `A` has its own `__new__` and class `B` inherits directly from `A` and uses `object.__new__(*b)` in its `__new__`. This doesn't raise any error when run in Python 3.3. Why Python raises the exception for builtin classes only, not user defined classes too? – GIZ Sep 15 '16 at 18:44

What is the downside of using object.__new__ in __new__?

2 Answers2

What is the downside of using object.new in new?