Import functions from dll (programmed in C++) into Python script when the dll has an embedded Python interpreter

Question

I wrote a library (dll) in c++ that uses an embedded Python interpreter. Executing Python scripts in the embedded interpreter works fine when the library is used by a c++ program.

If I call the functions of the dll from a Python program I run into trouble. When the dll would usually start the embedded interpreter by calling Py_Initialize() there is an interpreter running already (I can test this by calling Py_IsInitialized() ). When I try to use this interpreter the program crashes.

Minimal example that reproduces the behavior:

Code for dll:

#include "stdafx.h"
#include <Python.h>

extern "C" __declspec(dllexport)
int testIsInitialized()
{
  return Py_IsInitialized();
}

extern "C" __declspec(dllexport)
void testRunSimpleString()
{
  PyRun_SimpleString("print('test')");
}

Python script:

import ctypes

dll=ctypes.windll.LoadLibrary('pytest.dll')

# test if Python interpreter is initialized
print(dll.testIsInitialized())

# test if we can run a simple Python script
dll.testRunSimpleString()

Output:

1
Traceback (most recent call last):
  File "test_dll.py", line 9, in <module>
    dll.testRunSimpleString()
OSError: exception: access violation reading 0x0000000000000010

My main question is: How can I execute Python code from a c++ library that is imported by a different Python program?

UPDATE

The example works when I aquire the global interpreter lock (GIL) in my testRunSImpleString() function. I do not understand why this is necessary here.

Sooooo you have a c++ with a python in it, and you now want to use that python from within another python? Why? — Davesoft, Jul 20 '18 at 13:13
@Davesoft The dll should be usable from any programming language. Python is the special case where the dll does not work. I would prefer the execution of the embedded Python code in a separate interpreter. I am not sure how to achieve this. Maybe with a sub-interpreter? But the documentation for sub-interpreters is veryy thin. — siehe-falz, Jul 20 '18 at 13:35
Look at [`ctypes.PyDLL`](https://docs.python.org/2/library/ctypes.html#ctypes.PyDLL) - ctypes automatically releases the GIL and in your case this isn't helpful — DavidW, Jul 23 '18 at 16:59

Mike T · Answer 1 · 2019-08-07T21:45:32.870

The ctypes library has several classes to load different types of DLLs. Two relevant classes are:

ctypes.CDLL (i.e. C DLL) - used to load a DLL that uses a standard C calling convention. This includes C++ DLLs that define extern "C".
ctypes.PyDLL - similar to C DLL, except that the Python GIL is not released during the function call. Useful to call Python C API functions directly.

When loading a C++ DLL that uses the Python C API (i.e. Python.h), use ctypes.PyDLL:

import ctypes

dll = ctypes.PyDLL('pytest.dll')

# then do whatever
dll.testRunSimpleString.argtypes = []
dll.testRunSimpleString.restype = None
dll.testRunSimpleString()

One thing to keep in mind with PyDLL is to that you don't need to call Py_Initialize() or Py_Finalize()/Py_FinalizeEx() from the C++ API, since the interpreter is already running. You can first check (and store) Py_IsInitialized() to see if a new interpreter is need to be started/stopped.

Import functions from dll (programmed in C++) into Python script when the dll has an embedded Python interpreter

1 Answers1