Py_InitModule with multiple functions - invalid conversion from int to PyCFunction

Question

I am transitioning to removing the boost-python dependencies in my code, and I have made to to the "final step" of this transition (I removed all other boost dependencies, and when I comment out the boost code below I get ImportError: dynamic module does not define init function (initMy_module).

Here is the code as it stands

#include <boost/python.hpp>
namespace bpn = boost::python::numeric;
using namespace boost::python;
BOOST_PYTHON_MODULE(My_module)
{
  using namespace bpn;
  import_array();
  array::set_module_and_type("numpy", "ndarray");

  register_exception_translator<int>(&translator);

  def("run_My_module", run_My_module, "run my moduule");
  def("run_My_module_another_way", run_My_module_another_way, "run my module another way");
}

From my understanding of the python/C API, I believe that the following code should superficially link my C and Python code.

static PyMethodDef myModule_methods[] = {
    {"run_My_module", run_My_module, METH_VARARGS},
    {"run_My_module_another_way", run_My_module_another_way, METH_VARARGS},
    {NULL, NULL}
    };

void initmyModule(void)
{
    // Init module.
    (void) Py_InitModule("myModule", myModule_methods);

}

However, this produces the following errors:

myModule_python_binding.cpp:126:5: error: invalid conversion from 'int (*)(char*, char*, char*, PyObject*) {aka int (*)(char*, char*, char*, _object*)}' to 'PyCFunction {aka _object* (*)(_object*, _object*)}' [-fpermissive]
     };
     ^
myModule_python_binding.cpp:126:5: error: invalid conversion from 'int (*)(char*, ompi_communicator_t*&, char*, char*, PyObject*) {aka int (*)(char*, ompi_communicator_t*&, char*, char*, _object*)}' to 'PyCFunction {aka _object* (*)(_object*, _object*)}' [-fpermissive]

both functions are of this form. They both return integers indicating their level of success.

int run_myModule(char *infile, char *outfile, char *errfile, pyObject *exc)
{...};

int run_myModule_another_way(char *infile, int &_exvar, char *outfile, char *errfile, pyObject *exc)
{...};

Why is my "superficial" connection failing?

Your `run_My_module` and `run_My_module_another_way` functions have different signatures, which is a big hint of something being fishy. How do you expect them both to fit into the same array? Who would supply different parameters for them? Also you may actually read the error messages and try to understand what they are trying to tell. There seems to be a mention of``PyCFunction` in both of them, you may want to look that up. — n. m. could be an AI, Jun 21 '15 at 06:14
@n.m. can you explain how this code expects run_My_module and run_My_module_another_way to fit into the same array? I have been searching through PyMethodDef and I cannot find information on how to use Py_InitModule with multiple functions — kilojoules, Jun 22 '15 at 20:02
I have no idea what this code expects. IMO code is inanimate and has mo mind of its own so it cannot expect anything. The question is what **you** expected when you wrote these functions with signatures they have. No other code knows about these signatures. Who is going to call them? — n. m. could be an AI, Jun 23 '15 at 05:46
Perhaps you should not ignore the bit about PyCFunction either. Have you looked it up? — n. m. could be an AI, Jun 23 '15 at 05:52
@n.m. Yes I found the documentation for PyCFunciton. `Functions of this type take two PyObject* parameters and return one such value`. I am converting from boost. Does a boost python-callable function not have to follow this restrictive format? It is still not clear to me if I need to restructure the functions or how that would be accomplished. — kilojoules, Jun 23 '15 at 17:00
That's correct. Boost uses template magic to take care of necessary conversions. It takes any function and wraps it in a PyCFunction-compatible adapter behind the scenes. If you get rid of Boost you need to do it yourself. — n. m. could be an AI, Jun 24 '15 at 06:46

score 1 · Answer 1 · answered Jul 10 '15 at 13:00

You are getting the compile time error because the function pointers that you are using in myModule_methods[] are of the wrong type/signature. The functions that you want to call directly from Python need to have the following signature:

PyObject *foo(PyObject *self, PyObject *args)

So if you want to call your run_my_Module and run_my_Module_another_way functions from Python, you need to create the necessary "glue" between your functions and Python by wrapping them in a function that has the above signature. For example:

static PyObject *run_My_module_wrapper(PyObject *self, PyObject *args)
{
    // Parse arguments from Python out of the args parameter using
    // PyArg_ParseTuple. i.e. somthing like:
    const char *infile, *outfile, *errfile;
    PyObject *exc;

    if (!PyArg_ParseTuple(args, "sssO", &infile, &outfile, &errfile, &exc)) {
        return NULL; // Invalid arguments
    }

    //
    // Now you can call your function with the arguments from the Python call.
    //
    int r = run_My_module(infile, outfile, errfile, exc);

    if (r == SUCCESSFUL_VALUE) {
        Py_RETURN_NONE;
    }
    else {
        return NULL;
    }
}

// Similar setup for run_My_module_another_way.

static PyMethodDef myModule_methods[] = {
    {"run_My_module", run_My_module_wrapper, METH_VARARGS},
    {"run_My_module_another_way", run_My_module_another_way_wrapper, METH_VARARGS},
    {NULL, NULL}
};

void initmyModule(void)
{
    // Init module.
    (void) Py_InitModule("myModule", myModule_methods);

}

See Parsing Arguments for more info on parsing arguments passed in from Python. Plus Extending and Embedding the Python Interpreter for all of the general info you need.

Py_InitModule with multiple functions - invalid conversion from int to PyCFunction

1 Answers1