35

I'm a PhD student and use Python to write the code I use for my research. My workflow often consists of making a small change to the code, running the program, seeing whether the results improved, and repeating the process. Because of this, I find myself spending more time waiting for my program to run than I do actually working on it (a common experience, I know). I'm currently using the most recent version of Python 2 on my system, so my question is whether switching to Python 3 is going to give me any speed boost or not. At this point, I don't really have a compelling reason to move to Python 3, so if the execution speeds are similar, I'll probably just stick with 2.x. I know I'm going to have to modify my code a bit to get it working in Python 3, so it's not trivial to just test it on both versions to see which runs faster. I'd need to be reasonably confident I will get a speed improvement before I spend the time updating my code to Python 3.

Colin
  • 10,447
  • 11
  • 46
  • 54
  • 9
    How about profiling your code and writing the worst performing or most critical pieces in c? – Ross Rogers Jan 21 '10 at 19:46
  • 2
    This doesn't directly address your question, but depending on the nature of your software, you could try using Python's `pickle` module to easily store and load intermediate results that aren't affected by your changes. Using some "dummy data" gathered from an initial run will isolate your repetitive processing to just what you need to test your changes. – ezod Jan 21 '10 at 19:50
  • 7
    Also consider psyco ( http://psyco.sourceforge.net/ ) and unladen-swallow ( http://code.google.com/p/unladen-swallow/wiki/Release2009Q3 ). – Ross Rogers Jan 21 '10 at 19:51
  • 1
    I agree with Ross, Psyco is definitely your friend. Assuming you can use it (it won't run on 64-bit versions of Python), it's pretty much a free speed boost. 5 minutes to deploy and you'll have some joy. Psyco is not available for Python 3.0, I'm not sure there are any plans to make it so. – Brian Jan 21 '10 at 19:58
  • 1
    I'm planning on doing some profiling, I was just seeing if updating to Python 3 could be a quick and easy way to get some improvement. – Colin Jan 21 '10 at 20:00
  • Psyco looks promising, thanks! – Colin Jan 21 '10 at 20:01
  • What is your problem domain? There's bound to be people who know how to optimize it in Python. Also, have a look at the inbuilt profilers. They are bound to help. – wisty Jan 22 '10 at 04:30
  • possible duplicate of [Python 3 performance?](http://stackoverflow.com/questions/170426/python-3-performance) – ripper234 Mar 19 '11 at 12:48
  • Hi Colin, Did you ever make the switch to python 3.0+? The time to load modules seems to be as much or longer than the time to execute code for low level test cases. Was it module loading or code execution that created the most time sink? – dhj Oct 11 '12 at 06:12
  • dhj, I'm still using python 2.7. Code execution is by far the main time sink as I'm running genetic algorithm simulations. – Colin Oct 11 '12 at 16:07
  • Due to continuing development on Python 3, this question and its answers are too out of date to be usable, and will continue to get worse, since they are not being maintained. – Brian Oct 10 '14 at 21:09

5 Answers5

23

This article (archive.org) said that there were a few points where Python 3.0 was actually slower than Python 2.6, though I think many of these issues were resolved. That being said, Numpy hasn't been brought over to Python 3.0 yet and that's where a lot of the high performance (written in c) number functionality stuff is hiding. Hopefully it will be ready late 2009 or early 2010.

You should not consider performance to be a justification to switch to Python 3; I don't think you'll see a consistent speed improvement.

Edit: Versions of Numpy which support Python 3 have since been released.

Edit2: This answer (and other answers to this question) are outdated.

Brian
  • 25,523
  • 18
  • 82
  • 173
  • That's good enough for me. I'm new to Python so I wasn't sure what the goals were for Python 3, but speed doesn't seem to be one of them. – Colin Jan 21 '10 at 19:59
  • > Hopefully it will be ready late 2009 or early 2010. (it's already 2010 so I doubt they will hit a 2009 release date :) – Corey Goldberg Jan 21 '10 at 20:23
  • @Colin: True. To go a step further, speed was largely ignored. The philosophy is probably something like, "make it work, *then* make it faster." The goals for Python 3: http://wiki.python.org/moin/Python3.0 – Brian Jan 22 '10 at 19:04
5

Right now, speed on Python 3 is more or less the same as Python 2... If you're looking for speed, it's not on Python 3 vs Python 2 but in other tools like Psyco, Cython, etc...

But, very recently, there have arisen plans to merge Unladen Swallow, the Google project to implement a JIT over Python with Python 3. Of course, it won't be very soon, but, in some time, maybe the speed will increase noticeably on Python 3 over Python 2. They claim to have already increased speed on a 10% (on Python 2). Their objective is increasing the speed to 5x.

For more information, see PEP 3146

EDIT: Just as Brian remarks, PEP 3146 was retired.

Khelben
  • 6,283
  • 6
  • 33
  • 46
3

Try refining the algorithms or changing the data structures used. That's usually the best way to get an increase in performance.

2

I can't answer the root of your question, but if you read anything regarding the sluggish performance of the io module please disregard it. The were definitely performance issues in Python 3.0, but they were largely resolved in Python 3.1.

Eric Palakovich Carr
  • 22,701
  • 8
  • 49
  • 54
  • I did see something about that when I googling around. Good to know. – Colin Jan 21 '10 at 20:10
  • Eric, have you got a link to anything authoritative that you could add? I've still got the idea (after reading the "What's New" docs among other things) that although 3.1 improved things a lot, it was by no means a case of restoring the speed to where 2.5 was at. – Peter Hansen Jan 22 '10 at 04:08
  • I attended a meetup with Steve Holden and Eric Smith (chairman of the Python Software Foundation and contributor to the 3.1 changes respectively) and both said that things were much improved over 3.0 for the io module. I don't remember if they commented on how performance compared to the 2.x branch as it was a few months ago now. So I can't say 3.1's io module is better than 2.5, but I can say 3.1 is better than 3.0. – Eric Palakovich Carr Jan 22 '10 at 16:02
  • Thanks Eric, that's helpful. I notice that the `io` module appears to be getting ported to 2.7, so I guess one could easily compare the old and new I/O performance. See http://docs.python.org/dev/whatsnew/2.7.html – Peter Hansen Jan 22 '10 at 17:46
  • 2
    Stumbled over a link to this on Planet Python: http://www.dabeaz.com/blog/2010/01/reexamining-python-3-text-io.html It describes how text I/O in 3.1 is still quite a bit slower than in 2.6. Slower for good reason (Unicode) but still slower. Worth a read. – Peter Hansen Jan 29 '10 at 21:20
2

I have phylogenetics analysis that takes a long time to run, and uses about a half-dozen python scripts as well as other bioinformatics software (muscle, clustal, blast, even R!). I use temp files to save intermediate results and a master script with the subprocess module to glue all the pieces together. It's easy to change the master to run only the modified parts that I want to test. But, if the changes are being made to early steps, and you only know how good it is at the end of the whole process, then this strategy wouldn't help much.

telliott99
  • 7,762
  • 4
  • 26
  • 26