Interface C/C++ using different Python tools

Question

So I am doing a project that requires me to write some python codes to interface C/C++ programs, but I haven't decided which tool I should use. Intuitively, I'd like to choose between pybind11 and Boost.Python. Programmers: what are the pro and con between the twos? How would you decide which one to use?

Answer 1

Boost.Python was designed, as far as I can ascertain, with two primary goals in mind:

1. Offer pro-Python C++ developers a “pythonic” API in C++ for extension development – such that
2. … this API’s architects would try as hard as humanly possible to insulate its target developer user base from implementation details of the underlying Python C-API – specifically, they don’t want users to have to manipulate PyObject* values and reference counts.

As you might expect, Boost.Python uses Boost primitives, from myriad low-level Boost libraries, to accomplish this – which it does quite well:

1. if your project has already bought into using Boost,
2. for like 98% of the imaginable Python-to-C++ use-cases.

I personally dislike having to use Boost; if you are of a similar disposition and in a position to make executive decisions of this nature for your project, you can get what I believe is the overarching majority of the Boost.Python C++ API without the Boost-dependency overhead if you use pybind11 – a newer project that leverages C++11 features, reimplementing the low-level tools the pseudo-Pythonic API requires.

TL;DR: pybind11 is Boost.Python for developers with moderate to severe Boost allergies. Both of these tools make you write a C++ layer that exposes your existing API – which might be functions, types, templates, etc – as analagous Python structures, via one or more Python modules.

By contrast: Cython uses a different approach altogether. Cython provides a superset of the Python language, which additionally furnishes keywords that allow C and C++ primitives to be directly manipulated alongside related Python data structures. This is a totally different approach, and lends itself to two (often somewhat orthogonal) use-cases:

1. Optimizing Python code by judiciously moving a “hot” idiom out of the interpreted Python domain and into the native-compiled Cython domain; and
2. Providing a Python interface to a pre-existing C or C++ library.

… while these are different tasks, often one writes an extension in order to run something faster than one can in Python alone, and so using Cython for reason #1 can obviate the need for #2 (regardless of the tools one might use).

I like Cython, but I find its use for interfacing (use #2) improves when one knows C++ pretty cold. It is very easy to pick up Cython and use it to optimize some bottlenecked Python code (use #1) without being an expert C-hacker or C++ afficionato – but this, IMO, does not extend to Cythonic interfacing.

This is, of course, a highly subjective assessment – the asker is invited to do some sample programs with these frameworks, and draw his or her own conclusions.