How to define a new class with methods via the C API with Python 3?

Question

Short version: If we use PyObject_CallFunctionObjArgs with PyType_Type in Python 3 as a replacement for PyClass_New, then the dictionary has to be populated before the creation of the class. However, PyMethod_New requires the class to be known before the creation of the method. How can we add a method to the dictionary of a new class in Python 3?

In Python 2, I was used to define new classes with PyClass_New. Here is a minimal working example (for Python 2), where I define a new class example.MyClass with a method hello:

#include <Python.h>
#include <stdio.h>

PyObject *
callback(PyObject *obj, PyObject *args)
{
    printf("Hello world!\n");
    return Py_None;
}

int
main(int argc, char *argv[])
{
    Py_SetProgramName(argv[0]);
    Py_Initialize();
    PyObject *module = PyModule_New("example");
    PyDict_SetItemString(PyImport_GetModuleDict(), "example", module);
    PyObject *dict = PyDict_New();
    PyObject *classname = PyBytes_FromString("MyClass");
    PyObject *class = PyClass_New(PyTuple_New(0), dict, classname);
    PyMethodDef method_def;
    method_def.ml_name = "method_closure";
    method_def.ml_meth = callback;
    method_def.ml_flags = 1;
    method_def.ml_doc = "Method closure";
    PyObject *closure = PyCFunction_New(&method_def, NULL);
    PyObject *method = PyMethod_New(closure, NULL, class);
    PyDict_SetItemString(dict, "hello", method);
    PyModule_AddObject(module, "MyClass", class);
    PyRun_SimpleString("from example import MyClass\nMyClass().hello()");
    Py_Finalize();
    return 0;
}

PyClass_New was removed from the Python 3 C API. I read that PyObject_CallFunctionObjArgs can be used as replacement, for example in the answer of the following question: What is the PyClass_New equivalent in Python 3?

However, it appears that the dictionary that PyObject_CallFunctionObjArgs takes for last argument is copied during the creation of the new class: indeed, the fields that are set to dict before calling PyObject_CallFunctionObjArgs are available in the instances of the class, but not the fields that are set after the call. That prevents us from using PyMethod_New as above because the class has to be known before creating the method, but the method has to be added to dict before creating the class.

This dependency loop can be solved by using PyInstanceMethod_new, since the latter does not take a reference for a class. I obtained thus the following working example (for Python 3, I only give the code for main, the preambule and the callback is the same as in the first example).

int
main(int argc, char *argv[])
{
    wchar_t progname[FILENAME_MAX + 1];
    mbstowcs(progname, argv[0], strlen(argv[0]) + 1);
    Py_SetProgramName(progname);
    Py_Initialize();
    PyObject *module = PyModule_New("example");
    PyDict_SetItemString(PyImport_GetModuleDict(), "example", module);
    PyObject *dict = PyDict_New();
    PyMethodDef method_def;
    method_def.ml_name = "method_closure";
    method_def.ml_meth = callback;
    method_def.ml_flags = 1;
    method_def.ml_doc = "Method closure";
    PyObject *closure = PyCFunction_New(&method_def, NULL);
    PyObject *method = PyInstanceMethod_New(closure);
    PyDict_SetItemString(dict, "hello", method);
    PyObject *classname = PyUnicode_FromString("MyClass");
    PyObject *class = PyObject_CallFunctionObjArgs(
        (PyObject *) &PyType_Type, classname, PyTuple_New(0), dict);
    PyModule_AddObject(module, "MyClass", class);
    PyRun_SimpleString("from example import MyClass\nMyClass().hello()");
    Py_Finalize();
    return 0;
}

However, I wonder whether I could achieve to write a solution closer to the Python 2 version, using PyMethod_new: it should be possible to dynamically change the class after its creation for adding the new method, but I don't figure out how to do that.

score 1 · Answer 1 · answered May 17 '16 at 16:35

You're not actually using the C API for creating types.

The normal way to define a type through the C API is by defining your object's layout as a C struct and declaring a global PyTypeObject with tp_* fields defining its properties. This is documented under Extending and Embedding the Python Interpreter: Defining New Types. PyObject_CallFunctionObjArgs is not involved.

Using PyObject_CallFunctionObjArgs with PyType_Type would be analogous to creating a type by calling type manually in Python. In this case, the type's dict should not contain method objects. Just like if you were doing it in Python, the dict should contain Python function objects, and method objects will be created dynamically on attribute access.

How to define a new class with methods via the C API with Python 3?

1 Answers1