Crash calling C++ virtual member from Python extension

Question

I'm trying to write a thin wrapper around some C++ classes to call them from Python. The specific problem is that if virtual is uncommented below then attempting to create a Foo() crashes the interpreter. I'm not interested in rewriting this code as Boost::python, SWIG or Pyxxx to solve this problem - this is an extract from a much larger system I would like to know why it happens, although if either of those libraries solves this problem I would love to know how they implement it. If I setup an extension module and register types in the system then the calls from the Python interpreter work without any problems, either using extern "C" or just having static members in a class to register in the slots for the Python type.

If I declare a member to be virtual, then attempting to call it (within a call from the Python interpreter) causes a crash with bad memory access. The address accessed is the offset that I get if I print out the class member as a pointer. I have specific code to follow, but the basic question is: does something in the C runtime environment called by Python screw up the despatch of virtual class member functions? Python is v2.6.7 and the C++ extension is being compiled by GCC 4.2.1. There are related questions that suggest Boost::python supports this, are they doing it directly or simulating it through class member functions?

// PythonType is a class that inherits from PyTypeObject and fills in defaults ...

extern "C" int initHook(PyObject *self, PyObject *args, PyObject *kwds);
class Foo
{
public:
static PythonType thePyType;
  static void registerType(PyObject *module)
  {
    thePyType.tp_init = initHook;
    PyType_Ready(&thePyType);
    PyModule_AddObject(module, thePyType.tp_name, (PyObject*)&thePyType);
  }

  /*virtual*/ void insert()
  {
    printf("Inserted\n");
  }
};
PythonType Foo::thePyType("Foo",sizeof(Foo));
extern "C" {
  int initHook(PyObject *self, PyObject *args, PyObject *kwds)
  {
    ((Foo*)self)->insert();
  }
}

I have no idea how python binding work, but you definitely can't cast C++ object like this. — Šimon Tóth, Aug 02 '12 at 11:49

score 0 · Answer 1 · edited May 23 '17 at 12:29

0

Thanks Let_Me_Be, your comment pushed me down the right track. The problem is that the memory for the Foo object is being malloc'd by the Python interpreter. This hasn't caused problems in my other code because it does not rely on the vtable being initialised. The easiest solution is to use placement-new to ensure the ctor actually gets called and then the virtual member works correctly:

 int initHook(PyObject *self, PyObject *args, PyObject *kwds)
 {
    new(self) Foo();
    ((Foo*)self)->insert();
 }

There is a more detailed Q&A at On VTable pointers and malloc.

edited May 23 '17 at 12:29

Community

1
1

answered Aug 02 '12 at 12:15

Andrew

2,943
18
23

2

You're using placement new over a PyObject* and initializing another, completely unrelated type on that memory location? How is CPython supposed to manage that? – mfontanini Aug 02 '12 at 12:18
The PyTypeObject carries unknown info around with it, the size field is so that it can malloc enough space for this unknown structure. The PyObject* is not actually for a PyObject, it is for this unknown structure that has been allocated and then passed around as this type. Python extension code then casts the pointer to the real type to use it. – Andrew Aug 02 '12 at 13:06

Crash calling C++ virtual member from Python extension

1 Answers1