Building a Python C extension on Windows

Question

I'm trying to build a small Python module in C. It's just a single .C file that includes a single header file and links a single static library.

On Linux everything works as expected, but one of our team members uses Windows, so I'd like to build this module for Windows as well.

I'm using Windows 10 with Visual Studio Build Tools 2017 installed, including the VC++ 2015.3 v14.00 (v140) toolset for desktop. I'm using Python 3.6.

from setuptools import setup, Extension

drone_logger_module = Extension('DroneLogger',
                                include_dirs=['Output\\include'],
                                libraries=['DroneLoggerWindows'],
                                library_dirs=['x64\\Release'],
                                sources=['Output\\src\\DroneLogger.py.c'])

setup([...],
      ext_modules=[drone_logger_module])

When I ran python setup.py bdist_wininst, I got the following output:

running bdist_wininst
running build
running build_ext
building 'DroneLogger' extension
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IOutput\include -IC:\Users\piete\AppData\Local\Programs\Python\Python36\include -IC:\Users\piete\AppData\Local\Programs\Python\Python36\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" /TcOutput\src\DroneLogger.py.c /Fobuild\temp.win-amd64-3.6\Release\Output\src\DroneLogger.py.obj
DroneLogger.py.c
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:x64\Release /LIBPATH:C:\Users\piete\AppData\Local\Programs\Python\Python36\libs /LIBPATH:C:\Users\piete\AppData\Local\Programs\Python\Python36\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64" DroneLoggerWindows.lib /EXPORT:PyInit_DroneLogger build\temp.win-amd64-3.6\Release\Output\src\DroneLogger.py.obj /OUT:build\lib.win-amd64-3.6\DroneLogger.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\Output\src\DroneLogger.cp36-win_amd64.lib
DroneLogger.py.obj : warning LNK4197: export 'PyInit_DroneLogger' specified multiple times; using first specification
Creating library build\temp.win-amd64-3.6\Release\Output\src\DroneLogger.cp36-win_amd64.lib and object build\temp.win-amd64-3.6\Release\Output\src\DroneLogger.cp36-win_amd64.exp
Generating code
Finished generating code
LINK : fatal error LNK1158: cannot run 'rc.exe'
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed
with exit status 1158

I then followed this SO answer, and copied C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\rc.exe and rcdll.dll to C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\. This solved the linking problem.

I've spent days trying to get it to work on Windows, and the solution I came up with is kind of questionable (that is, I don't know why it doesn't work out of the box, and I have a feeling that I shouldn't be moving Microsoft EXEs and DLLs around just to get it working).

My question is twofold:

Is there really no better/easier way to build a Windows installer for a Python C extension?
Why does this solution work, and why is it necessary?

I would expect it to be a dll (or "pyd"). But for building, you might benefit from one of the batch files which sets up a command line shell with the path and environment variables. VsDevCmd.bat or vsvars32.bat for example, in the tools folder. — Kenny Ostrom, Dec 15 '18 at 19:15
I'm not sure I understand your comment, could you elaborate? — tttapa, Dec 15 '18 at 21:46
I was saying visual studio usually is packaged with scripts (.bat files) that set up the build environment. My comment may not apply to your case. — Kenny Ostrom, Dec 16 '18 at 15:03

score 3 · Answer 1 · answered Dec 16 '18 at 12:35

What might make it generally easier to compile Python extensions on Linux than on Windows is that with standard package management on Linux you will generally have the same C compiler installed that was used to compile Python itself. On Windows, you have the choice of several Visual Studio versions, mingw, cygwin. For Windows, I find it most straightforward to use the special Visual C++ for Python 2.7 package from Microsoft, which, unlike the newer Visual Studio version used in the question asked, has rc.exe properly in its path together with the Visual C compiler. Another solution is to install the mingw compilers and create a file distutils.cfg in Lib/distutils in the main python folder with the contents

[build]
compiler=mingw32

[build_ext]
compiler=mingw32

to use these compilers instead. Unless -static-libgcc (and -static-libstdc++ for C++ projects) is added to the linker flags, the DLLs for libgcc (and for libstdc++) must be shipped with the extension so it will work also without the mingw runtime installed/in path.

Building a Python C extension on Windows

1 Answers1