How can I build manually C++ extension with mingw-w64, Python and pybind11?

Question

My final goal is to compile Python C++ extension from my C++ code. Currently to get started I am following a simple example from the first steps of pybind11 documentation. My working environment is a Windows 7 Professional 64 bit, mingw-w64 (x86_64-8.1.0-posix-seh-rt_v6-rev0) and Anaconda3 with Python 3.7.4 64 bit. I have 2 files. The first one is a C++ file -- example.cpp

#include <pybind11/pybind11.h>

int add(int i, int j) {
    return i + j;
}

PYBIND11_MODULE(example, m) {
    m.doc() = "pybind11 example plugin"; // optional module docstring

    m.def("add", &add, "A function which adds two numbers");
}

I compile the C++ file with the following command:


C:/mingw-w64/x86_64-8.1.0-posix-seh-rt_v6-rev0/mingw64/bin/g++.exe -shared -std=c++11 -DMS_WIN64 -fPIC -ID:\Users\ADAS\anaconda3\Include -ID:\Users\ADAS\anaconda3\Library\include -ID:\Users\ADAS\anaconda3\pkgs\python-3.7.4-h5263a28_0\include -Wall -LD:\Users\ADAS\anaconda3\Lib -LD:\Users\ADAS\anaconda3\pkgs\python-3.7.4-h5263a28_0\libs example.cpp -o example.dll -lPython37

The compilation result is successful and I am getting example.dll file.

At the next step I run the following Python code -- example.py:

import example

def main():
    i, j = (1, 2)
    res = example.add(i, j)
    print("%d + %d = %d" % (i, j, res))

if __name__ == '__main__':
    main()

And here I have got an issue. It seems the import example line does not give me any warnings or errors but the line res = example.add(i, j) gives me an error:

AttributeError: module 'example' has no attribute 'add'

Under Ubuntu 18.04 I successfully compiled and run in Python the example above but in my office I have only Windows 7.

Questions: what is wrong in either my setup or command line? Is it possible to fix this problem without changing current C++ compiler (mingw-w64 version 8.1) under Windows?

Vladimir S. · Accepted Answer · 2023-03-10T02:56:54.493

It is unbelievable! The problem was just the file extension of the compiled file. As soon as I changed .dll to .pyd, the Python example (example.py) is running without any issue!

So the new command line is:

C:/mingw-w64/x86_64-8.1.0-posix-seh-rt_v6-rev0/mingw64/bin/g++.exe -shared -std=c++11 -DMS_WIN64 -fPIC -ID:\Users\ADAS\anaconda3\Include -ID:\Users\ADAS\anaconda3\Library\include -ID:\Users\ADAS\anaconda3\pkgs\python-3.7.4-h5263a28_0\include -Wall -LD:\Users\ADAS\anaconda3\Lib -LD:\Users\ADAS\anaconda3\pkgs\python-3.7.4-h5263a28_0\libs example.cpp -o example.pyd -lPython37

Because I did some experiments with the command line arguments, I am going to check again all the compiler arguments to make sure it gives the successful result. I will let you know, if some changes are still required.

Update1:

According to Python3 default settings, the full extension of the compiled C++ file under Windows has to be .cp37-win_amd64.pyd.

We can get the extension by terminal command:

python -c "from distutils import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))"

That is an equivalent of the python3-config --extension-suffix from pybind11 documentation. The python3-config script is not implemented in Windows environment (at least in Anaconda3 distribution).

Update2:

Under Linux OS (tested in Ubuntu 20.04), the command line is like

c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)

I am able to generate my binding with the .cp39-win_amd64.pyd extension. However, when I try to import it I am getting the `ModuleNotFoundError`.... any pointers you can give? I tried the `os.add_dll_directory()` as well. — tushaR, Feb 26 '23 at 01:23
@tushaR I have seen your question in new thread, that answer you have got is absolutely correct: a name of C++ file of the Python extension must be a name for Python module to import from, in my case it is: `example.cpp` --> `example.cp37-win_amd64.pyd` --> `import example` in Python code. — Vladimir S., Feb 28 '23 at 06:08

How can I build manually C++ extension with mingw-w64, Python and pybind11?

1 Answers1

Linked