How does pypy handle recursions?

Question

I have a script I wrote in python and it works fine but I was curious to see if I could speed it up. It basically is recursive script.

If I run it within normal python 2.7, it takes about 30 seconds. When I run the same thing using pypy than I get the following error:

RuntimeError: maximum recursion depth exceeded

I'm not sure what pypy is doing differently because I'm not modifying the script.

Can anyone help me understand what is going on?

Update: ok I figured it out. Increasing the limit helped but I think I was running the wrong file. I found a file under the bin directory called py.py and was using that. I'm not sure what the file does but its slower than normal python. I had to search and find 'pypy-c' seems to work now.

Without seeing the code, it would be hard to diagnose. The recursion limit is an implementation-specific limit, PyPy might have a smaller one than CPython? How many levels of recursion are you expecting? — Ned Batchelder, Sep 24 '11 at 18:25
Seeing the source code might help. If it's sufficently short, or can be made so while preserving this behaviour (works on CPython, recursion error on PyPy), it may also be worthwile submitting a PyPy bug report, as compability with CPython is high priority for PyPy. I don't think it's likely that this falls under one of the few intended/accepted incompabilities. — , Sep 24 '11 at 18:26
@NedBatchelder: Good idea, agreed. Although http://pypy.readthedocs.org/en/latest/cpython_differences.html#miscellaneous suggests that PyPy's recursion limit is higher than CPython's default (I think the exact depth permitted should depend on the size of a call frame, so if you use far more variables than an average function, the limit may be lower). — , Sep 24 '11 at 18:29
Thank you, Deinan and Ned. I updated my code but still getting the same problem despite following DSM's advice). The only thing I think I did differently is I complied from source(since no binaries exist for my distribution). I do get some weird c messages when I start it on my linux distro versus running it on my mac(installed via binaries) but the demo files for pypy work under both installs so I'm not sure if thats a problem. — Lostsoul, Sep 24 '11 at 18:52
I just tried this on my copy of PyPy and it runs correctly and quickly. What version of PyPy are you using? — Winston Ewert, Sep 24 '11 at 19:12
Ah, the updates clears things up. You were running PyPy interpreted under CPython, which naturally is much slower and pushes the 1000-call limit of CPython with both the interpreter's and your code's calls. `pypy-c` is the natively-compiled interpreter that's intended to be used, and it handles recursion as well as expected. — , Sep 24 '11 at 19:14
@delnan: looks like you beat me by 1 minute to the py.py issue. Since the setrecursionlimit issue was a detour I've deleted that answer (manually testing it gives me 1010, which is far more sensible.) — DSM, Sep 24 '11 at 19:20
Thank you so much guys..I'm really sorry for such a newbie mistake. I'm a bit new to programming figuring out new software. Thanks again. — Lostsoul, Sep 25 '11 at 00:55

score 5 · Accepted Answer · answered Sep 24 '11 at 19:22

As you suggest in your update your problem was that you were using py.py (which is for running PyPy's interpreter on top of CPython). PyPy has a higher recursion limit than CPython normally. You can use sys.setrecursionlimit() to increase the recursion limit, sys.getrecursionlimit() does not provide the actual recursion limit.

PyPy 1.6.0:

>>>> sys.getrecursionlimit()
100

>>>> def infinite(level=0):
....     print level
....     return infinite(level+1)
.... 

>>> infinite()
<snip>
1010
Traceback (most recent call last):
  File "<console>", line 2, in infinite
RuntimeError: maximum recursion depth exceeded

>>> sys.setrecursionlimit(sys.maxint)

>>> infinite()
<snip>
9769
zsh: segmentation fault  pypy

How does pypy handle recursions?

1 Answers1