Implementing a function in Python vs C

Question

Is there a difference (in terms of execution time) between implementing a function in Python and implementing it in C and then calling it from Python? If so, why?

Eh, what? For starters, you (generally) don't compile Python and need a wrapper anyway to call C functions from Python or vice versa. For slightly advanced users, do you realize that performance heavily depends on the algorithm, then on the implementation (both of the code and the language implementation), not on the language alone? (You can use LLVM/Clang to *interpret* C and C++... and several projects to vastly speed Python code up through compilation). Not to mention that many wouldn't be able to get a working C version within acceptable time. — , Feb 11 '11 at 11:04

Tim Pietzcker · Accepted Answer · 2011-02-11T11:13:01.950

5

Python (at least the "standard" CPython implementation) never actually compiles to native machine code; it compiles to bytecode which is then interpreted. So a C function which is in fact compiled to machine code will run faster; the question is whether it will make a relevant difference. So what's the actual problem you're trying to solve?

edited Feb 11 '11 at 11:13

answered Feb 11 '11 at 11:03

Tim Pietzcker

328,213
58
503
561

3

To be pedantic, Python can be compiled - JIT-compilation in Jython, IronPython and PyPy (and Psyco on x86), AOT-compilation in Cython (okay, strictly speaking a different language) and for some code through various projects in various stages of completition (e.g. Shedskin. – Feb 11 '11 at 11:09
I want to implement The levenberg marquardt algorithm – JohnJohnGa Feb 11 '11 at 11:15
3

@JohnJohnGa: Well, do you **know definitely** that a Python implementation will never be fast enough to be practical? If not, follow the useful rule "make it run, make it correct, make it fast - in that order" and write it in Python before optimizing (e.g. rewriting in C or Cython, although you can easily get 20x or more out of a sloppily-coded Python function). – Feb 11 '11 at 11:18
1

Well, there already exists [a GPL version in C](http://www.ics.forth.gr/~lourakis/levmar/), complete with [Python bindings](http://trac.astrometry.net/wiki/PyLevmar). I would expect this to be much faster than a native CPython implementation. – Tim Pietzcker Feb 11 '11 at 11:38

score 1 · Answer 2 · answered Feb 11 '11 at 11:10

If I understand and restate your question properly, you are asking, if wrapping python over a c executable be anyway faster than a pure python module itself? The answer to it is, it depends upon the executable and the kind of task you are performing.

There are a set of modules in Python that are written using Python C-API's. Performance of those would be comparable to wrapping a C executable
On the other hand, wrapping c program would be faster than pure python both implementing the same functionality with a sane logic. Compare difflib usage vs wrapping subprocess over diff.

score 1 · Answer 3 · edited Dec 15 '22 at 21:01

1

The C version is often faster, but not always. One of the main points of speedup is that C code does not have to look up values dynamically, like Python (Python has reference semantics). A good example for this is Numpy. Numpy arrays are typed, all values in the array have the same type, and are internally stored in continuous block of memory. This is the main reason that numpy is so much faster, because it skips all the dynamic variable lookup that Python has to do. The most efficient C implementation of an algorithm can become very slow if it operates on Python data structures, where each value has to be looked up dynamically.

A good way to implement such things yourself and save all the hassle of Python C-APIs is to use Cython.

edited Dec 15 '22 at 21:01

Glorfindel

21,988
13
81
109

answered Feb 11 '11 at 11:11

Björn Pollex

75,346
28
201
283

Homogeneous arrays actually are a bad example for the cost of dynamic member lookup, as completely unrelated. The cost comes from the indirection that's necessary because the items can be of different sizes (usually, C code interfacing with Python has to use pointers to `PyObject` structs). Thumbs up vor Cython though. – Feb 11 '11 at 11:16
@delnan: The example with the arrays was intended in contrast to Pythons own lists, and there the speedup is huge. The main point is using static typing. Btw, I actually learned about Cython from an answer you gave to [this question](http://stackoverflow.com/questions/4239371/) of mine :). – Björn Pollex Feb 11 '11 at 11:20

score 0 · Answer 4 · answered Feb 11 '11 at 11:03

Typically, a function written in C will be substantially faster that the Python equivalent. It is also much more difficult to integrate, since it involves:

compiling C code that #includes the Python headers and exposes appropriate wrapper code so that it is callable from Python;
linking against the correct Python libraries;
deploying the resulting shared library to the appropriate location, so that your Python code can import it.

You would want to be very certain that the benefits outweigh the costs before trying this, which means this should only be reserved for performance-critical sections of your code that you simply can't make fast enough with pure Python.

If you really need to go down this path, Boost.Python can make the task much less painful.

Implementing a function in Python vs C

4 Answers4