Using Python3 C API to add to builtins

Question

I'm looking to use the Python3 C API to add a builtin function. I'm doing this merely as an exercise to help me familiarize myself with the Python C API. The answer to this question does a pretty good job of explaining why one might not want to do this. Regardless, I want to add a function foo to the Python builtins module.

Here's what I've done so far (foo.c):

#include <Python.h>
#include <stdio.h>

static PyObject*
foo(PyObject *self, PyObject *args){
    printf("foo called");
    return Py_None;
}

char builtin_name[] = "builtins";
char foo_name[] = "foo";
char foo_doc[] = "foo function";

static PyMethodDef foo_method = {foo_name, foo, METH_NOARGS, foo_doc};

PyMODINIT_FUNC
PyInit_foo(void){
    PyObject *builtin_module = PyImport_ImportModule(builtin_name);
    PyModule_AddFunctions(builtin_module, &foo_method);
    return builtin_module;
}

I'm placing this in the Modules/ directory in the Python source directory.

Answer 1

Just because you put it in the Modules/ folder and use the Python-C-API doesn't mean it will be compiled and executed automagically. After you compiled your foo.c to a Python extension module (you did, right?) your code is (roughly) equivalent to:

foo.py

def foo():
    """foo function"""
    print("foo called")

import builtins
builtins.foo = foo

What isn't that straightforward in the Python implementation is the fact that when you import foo it won't return your foo module but builtins. But I would say that's not a good idea at all, especially since the builtin function you want to add has the same name as the module you created, so it's likely that by import foo you actually overwrite the manually added builtins.foo again...

Aside from that: Just by putting it in the Modules/ folder doesn't mean it's actually imported when you start Python. You either need to use import foo yourself or modify your Python startup to import it.

Okay, all that aside you should ask yourself the following questions:

• Do you want to compile your own Python? If yes, then you can simply edit the bltinsmodule.c in the Python/ folder and then compile Python completely.
• Do you even want to compile anything at all but not the complete Python? If yes, then just created your own extension module (essentially like you did already) but don't put it in the Modules/ folder of Python but really create a package (complete with setup.py and so on) and don't return the builtins module inside the module-init. Just create an empty module and return it after you added foo to the builtins module. And use a different module name, maybe _foo so it doesn't collide with the newly added builtins.foo function.
• Is the Python-C-API and an extension module the right way in this case? If you thought the Python-C-API would make it easier to add to the builtins then that's wrong. The Python-C-API just allows faster access and a bit more access to the Python functionality. There are only few things that you can do with the C-API that you cannot do with normal Python modules if all you want is to do Python stuff (and not interface to a C library). I would say that for your use-case creating an extension module is total overkill, so maybe just use a normal Python module instead.

My suggestion would be to use the foo.py I mentioned above and let Python import it on startup. For that you put the foo.py file (I really suggest you change the name to something like _foo.py) in the directory where the additional packages are installed (site-packages on windows) and use PYTHONSTARTUP (or another approach to customize the startup) to import that module on Python startup.