boost python runtime errors for what seem to be compile time issues - how effect performance?

Question

With boost python, I was adding an attribute to my python wrapper where the value came from an enumerated type, for instance:

scope().attr("myconstant")=some_namespace::some_class::some_enum_value;

But I got a run time error when I imported my python module:

terminate called after throwing an instance of 'boost::python::error_already_set'

Following other threads, I put the above in a try/catch but didn't catch anything to call PyErr_Fetch on. I'm still curious where the original python error occurred.

It turns out that I have to do

scope().attr("myconstant")=int(some_namespace::some_class::some_enum_value);

And then it runs.

Another, but I think related problem, is if you export a C++ function in your python wrapper that returns a C++ enum, but you do not export that enum, all is fine until you call this function from python. Then boost generates a python exception about a type not being found.

So clearly boost is doing some things at runtime that seem (to me) like they should be done at compile time. Both these problems were time consuming to diagnose. Does anyone know what is going on? With more things happening at runtime then I'd expect, will I hit performance issues with boost python that I wouldn't get if I worked directly with the python extension API? In addition to performance, I'm concerned that boost python code has more errors that won't be found until runtime then direct python extension code would have.

On the flip side, is there some big gain to be had with all this dynamic type binding? Clearly there is the nice boost interface for writing my own python extension - but does all this dynamic binding make it easier to add new boost python wrappers into an existing system then wrappers written directly with the python extension API?

Answer 1

With Python supporting dynamic name resolution, I would imagine it would be rather difficult and encumbering on the user for Boost.python to enforce compile-time assertions. The C++ code is exposed to Python at runtime, and it is possible that Boost.Python type conversions have been registered in a different module, loaded separately at runtime.

Boost.Python, and even the Python C-API, performs the majority of work at runtime. Boost.Python will not delay errors to runtime anymore than that of the Python C-API. When Python imports a Boost.Python module, the BOOST_PYTHON_MODULE code executes, creating types, registering type identities, and registering functions that allow for the seamless interoperability between Python and C++ objects. Overall, the same setup procedures would still be required with the C-API or any other Python language binding toolset. As Boost.Python provides a high-level general solution, one can expect it to be more bloated when compared to hand-written bindings tailored to the application.

My personal experience has been that it is easier to extend an existing system with Boost.Python rather than writing directly to the Python C-API. I have to write significantly less code, resulting in fewer errors while increasing readability and maintainability. For example, consider the official Python Extending Simple Example:

#include <stdlib.h> // system
#include <Python.h>

static PyObject*
spam_system(PyObject* self, PyObject* args)
{
  const char* command;
  if (!PyArg_ParseTuple(args, "s", &command))
    return NULL;
  return Py_BuildValue("i", system(command));
}

static PyMethodDef SpamMethods[] =
{
  {"system", &spam_system, METH_VARARGS,
   "Execute a shell command."},
  {NULL, NULL, 0, NULL} /* Sentinel */
};

PyMODINIT_FUNC
initspam(void)
{
  Py_InitModule("spam", SpamMethods);
}

and the equivalent in Boost.Python:

#include <stdlib.h> // system
#include <boost/python.hpp>

BOOST_PYTHON_MODULE(spam)
{
  boost::python::def("system", &system);
}