3

Something crossed my mind recently in Python: x = y(z) is equivalent to x = y.__call__(z). However, a test appears to invalidate that assumption and also leads to Python's interpreter to crash.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def ret(*args):
...     return args
...
>>> ret(1, 2, 3)
(1, 2, 3)
>>> for _ in range(1000000):
...     ret = ret.__call__
...
>>> ret(1, 2, 3)

Running the second ret(1, 2, 3) leads Python to crash and return to the command prompt (image).

  1. What is happening in the background when the line ret = ret.__call__ executes?
  2. Why does Python stop working on the last line, and should it be reported as a bug?

Useless Reference: Python functions and their __call__ attribute

Community
  • 1
  • 1
Noctis Skytower
  • 21,433
  • 16
  • 79
  • 117
  • 1
    I can't reproduce it in 2.7 or in 3.2.3. – senderle Dec 04 '12 at 22:00
  • did you mistyped? may be you mean `ret = ret.__call__(_)` – akaRem Dec 04 '12 at 22:03
  • @akaRem: That would not work at all. – Martijn Pieters Dec 04 '12 at 22:03
  • 1
    Please define "crash". I can reproduce(I think) on windows with python2.7/3.2 and 3.3. Anyway everytime you do `ret = ret.__call__` you are actually creating a new method-wrapper instance, and when the call is finally executed the interpreter has to pass thorugh all the wrappers you just created, so the function call would take forever to even start(even thouhg I'd assume it shouldn't take minutes... at least, usually you can do millions or more function calls for a minute) – Bakuriu Dec 04 '12 at 22:06
  • 1
    Sort of confirmed: [segmentation fault](http://ideone.com/0Irccz), though with different version of python. Same locally for 3.2. – gorlum0 Dec 04 '12 at 22:20

1 Answers1

1

You are creating a deeply nested structure of method wrappers. Each method wrapper still has a reference to self, where self is a reference to the parent method wrapper, all the way back to the original function:

>>> ret, ret.__call__.__self__
(<function ret at 0x10f17a050>, <function ret at 0x10f17a050>)
>>> ret.__call__, ret.__call__.__call__.__self__
(<method-wrapper '__call__' of function object at 0x10f17a050>, <method-wrapper '__call__' of function object at 0x10f17a050>)

Note how the memory address of the method wrapper __self__ attribute points to the parent object.

If you create enough of these wrappers, you could run out of memory.

Python creates such wrappers for all functions bound to an instance. It's the same for a custom class with a method:

>>> class Foo: 
...     def bar(self): return
... 
>>> Foo().bar
<bound method Foo.bar of <__main__.Foo object at 0x10f1798d0>>
>>> Foo().bar, Foo().bar.__self__
(<bound method Foo.bar of <__main__.Foo object at 0x10f179710>>, <__main__.Foo object at 0x10f179850>)

Methods are created from functions as needed, when accessing the method via attribute access. Because they hold a reference to self, they are retained in memory as long as you hold a reference to them. Your chain of references thus holds 100000 memory wrappers alive.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Thank you! I added an image to question to show what the computer was telling me. What I am confused about is why Python does not generate a `RuntimeError` and instead just quits. – Noctis Skytower Dec 04 '12 at 22:31