How to expose raw byte buffers with Boost::Python?

Question

I've got third party C++ library in which some class methods use raw byte buffers. I'm not quite sure how to deal in Boost::Python with it.

C++ library header is something like:

class CSomeClass
{
  public:
      int load( unsigned char *& pInBufferData, int & iInBufferSize );
      int save( unsigned char *& pOutBufferData, int & iOutBufferSize );
}

In stuck with the Boost::Python code...

class_<CSomeClass>("CSomeClass", init<>())
    .def("load", &CSomeClass::load, (args(/* what do I put here??? */)))
    .def("save", &CSomeClass::save, (args(/* what do I put here??? */)))

How do I wrap these raw buffers to expose them as raw strings in Python?

Answer 1

You have to write, yourself, functions on your bindings that will return a Py_buffer object from that data, allowing your to either read-only (use PyBuffer_FromMemory) or read-write (use PyBuffer_FromReadWriteMemory) your pre-allocated C/C++ memory from Python.

This is how it is going to look like (feedback most welcome):

#include <boost/python.hpp>

using namespace boost::python;

//I'm assuming your buffer data is allocated from CSomeClass::load()
//it should return the allocated size in the second argument
static object csomeclass_load(CSomeClass& self) {
  unsigned char* buffer;
  int size;
  self.load(buffer, size);

  //now you wrap that as buffer
  PyObject* py_buf = PyBuffer_FromReadWriteMemory(buffer, size);
  object retval = object(handle<>(py_buf));
  return retval;
}

static int csomeclass_save(CSomeClass& self, object buffer) {
  PyObject* py_buffer = buffer.ptr();
  if (!PyBuffer_Check(py_buffer)) {
    //raise TypeError using standard boost::python mechanisms
  }

  //you can also write checks here for length, verify the 
  //buffer is memory-contiguous, etc.
  unsigned char* cxx_buf = (unsigned char*)py_buffer.buf;
  int size = (int)py_buffer.len;
  return self.save(cxx_buf, size);
}

Later on, when you bind CSomeClass, use the static functions above instead of the methods load and save:

//I think that you should use boost::python::arg instead of boost::python::args
// -- it gives you better control on the documentation
class_<CSomeClass>("CSomeClass", init<>())
    .def("load", &csomeclass_load, (arg("self")), "doc for load - returns a buffer")
    .def("save", &csomeclass_save, (arg("self"), arg("buffer")), "doc for save - requires a buffer")
    ;

This would look pythonic enough to me.