How can I import an external C function into an IPython Notebook using Cython?

Question

I'd like to import a C function into an IPython notebook using Cython. Currently, I'm trying to replicate the example in the Cython documentation, but I get a compilation error.

My Python code (from an iPython notebook):

import cython
%load_ext Cython

---------------------------------- new cell

%%cython
cdef extern from "spam.c":
    void order_spam(int tons)

My C code:

// spam.c
#include <stdio.h>

static void order_spam(int tons)
{
    printf("Ordered %i tons of spam!\n", tons);
}

Running this code, I get the following traceback and error message:

CompileError                              Traceback (most recent call last)
<ipython-input-13-8bb733557977> in <module>()
----> 1 get_ipython().run_cell_magic(u'cython', u'', u'\ncdef extern from "spam.c":\n    void order_spam(int tons)')

/Users/danielacker/anaconda2/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_cell_magic(self, magic_name, line, cell)
   2118             magic_arg_s = self.var_expand(line, stack_depth)
   2119             with self.builtin_trap:
-> 2120                 result = fn(magic_arg_s, cell)
   2121             return result
   2122 

<decorator-gen-126> in cython(self, line, cell)

/Users/danielacker/anaconda2/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
    191     # but it's overkill for just that one bit of state.
    192     def magic_deco(arg):
--> 193         call = lambda f, *a, **k: f(*a, **k)
    194 
    195         if callable(arg):

/Users/danielacker/anaconda2/lib/python2.7/site-packages/Cython/Build/IpythonMagic.py in cython(self, line, cell)
    276             build_extension.build_temp = os.path.dirname(pyx_file)
    277             build_extension.build_lib  = lib_dir
--> 278             build_extension.run()
    279             self._code_cache[key] = module_name
    280 

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in run(self)
    337 
    338         # Now actually compile and link everything.
--> 339         self.build_extensions()
    340 
    341     def check_extensions_list(self, extensions):

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in build_extensions(self)
    446 
    447         for ext in self.extensions:
--> 448             self.build_extension(ext)
    449 
    450     def build_extension(self, ext):

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in build_extension(self, ext)
    496                                          debug=self.debug,
    497                                          extra_postargs=extra_args,
--> 498                                          depends=ext.depends)
    499 
    500         # XXX -- this is a Vile HACK!

/Users/danielacker/anaconda2/lib/python2.7/distutils/ccompiler.pyc in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends)
    572             except KeyError:
    573                 continue
--> 574             self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
    575 
    576         # Return *all* object filenames, not just the ones we just built.

/Users/danielacker/anaconda2/lib/python2.7/distutils/unixccompiler.pyc in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
    120                        extra_postargs)
    121         except DistutilsExecError, msg:
--> 122             raise CompileError, msg
    123 
    124     def create_static_lib(self, objects, output_libname,

CompileError: command 'gcc' failed with exit status 1

I've tried searching Google for this error, but I can't seem to find anything relevant.

Answer 1

The %cython magic command probaby doesn't seem to do the trick.

The %cython magic command doesn't really do the task here. In order to compile this you need to also supply the *.c source file and that (as far as I'm aware) isn't allowed with %cython. (The source file for it indicates that it only uses the text entered in the cell as a source file.)

Wrap your C functions:

Before presenting a probable solution, let me point out that the .pyx file you have created doesn't actually wrap the C function order_spam automatically. You can have it automatically wrapped if you specify it as cpdef inside the cdef extern block (alternatively you could wrap it on your own outside the cdef extern block, this allows more flexibility).

I'll use a filename of cyspam.pyx for the Cython file:

cdef extern from "spam.c":
    cpdef void order_spam(int tons)

Notice how I've prefixed the function declaration with cpdef, this instructs Cython to automatically wrap the function.

Use a setup.py script:

In order to have full control over the compilation process, you usually need to create a setup.py script that has all the required sources, include directories et cetera, specified.

Here's how the setup.py script should look like:

from distutils.core import setup, Extension
from Cython.Build import cythonize

# you specify the c source file in the sources list
ext = Extension('cyspam', sources = ['cyspam.pyx', 'spam.c'])
setup(name="C spam", ext_modules = cythonize([ext]))

You can either create a file like this via a simple text editor or via IPython with the %%writefile magic command. The setup.py script should of course be placed in the same directory as the cyspam.pyx and spam.c files.

Compile the files:

You can either open a terminal for this or use the %%bash command from IPython, either way does the trick. Issue the following command:

python setup.py build_ext --inplace 

--inplace places the generated .so file in the current directory.

After doing these you can easily import the file cyspam in Ipython and just call the wrapped C function:

Rounding up all commands from IPython:

All in all, if you want to do it only from IPython you'll issue the following commands:

In [1]: %%writefile setup.py
   ....: from distutils.core import setup, Extension
   ....: from Cython.Build import cythonize
   ....: ext = Extension('cyspam', sources = ['cyspam.pyx', 'spam.c'])
   ....: setup(name="C spam", ext_modules = cythonize([ext]))

In [2]: %%bash
   ...: python setup.py build_ext --inplace

In [3]: import cyspam

In [4]: cyspam.order_spam(1000)
You ordered 1000 ammount of spam!

---

As an alternative you could always create a .pyxbld file that specifies the needed arguments for pyximport.install(). This offers the same level of control but is most likely counter-intuitive for Python users who already have experience with using setup.py scripts.

See Related:

How can I set Cython compiler flags when using pyximport?