1

I'm trying to load AND call functions from an already compiled (not by me) C++ DLL file.

I'm faced with two problematic scenarios:

  1. Loading DLL by using CDLL():

    from ctypes import *
    __path_aa_Fn_dll__ = r'C:\user\aacad_2017\aa_Functions.dll' # DLL file path
    aa_Fn_dll = CDLL(__path_aa_Fn_dll__) # load DLL
    

    which gives me the module not found OSError:

    Traceback (most recent call last):
    File "C:/user/aacad_2017/cadClass.py", line 189, in <module>
    aa_Fn_dll = CDLL(__path_aa_Fn_dll__)
    File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
    OSError: [WinError 126] The specified module could not be found
    
  2. Loading the same DLL by using win32api.LoadLibraryEx() then CDLL()

    aa_Fn_dll_handle = win32api.LoadLibraryEx(__path_aa_Fn_dll__, 0, win32con.LOAD_LIBRARY_AS_DATAFILE) # get DLL handle
    aa_Fn_dll = CDLL(__path_aa_Fn_dll__, handle=aa_Fn_dll_handle) # load DLL
    

    which is able to load the DLL.

    However, when attempting to call functions within this DLL, the following AttributeError is raised:

    aa_Fn_dll.aaBIN2iv.restype = c_double # set function return type
    aa_Fn_dll.aaBIN2iv.argtypes = [c_double, c_double, c_double, c_double, c_double, c_double, c_double, c_char_p] # set function argument types
    aa_Fn_dll.aaBIN2iv(10, 10, 10, 10, 10, 10, 10, b"C") # call DLL function
    
    
    Traceback (most recent call last):
    File "C:/user/aacadMatlabWrap_2017_new_test/cadClass.py", line 196, in <module>
    aa_Fn_dll.aaBIN2iv.restype = c_double
    File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\ctypes\__init__.py", line 361, in __getattr__
    func = self.__getitem__(name)
    File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\ctypes\__init__.py", line 366, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
    AttributeError: function 'aaBIN2iv' not found
    

After extensive online research and consulting the Ctypes documentation, I have tried the following historical solutions:

  • Use Dependency Walker to check dependencies
  • Ensure all DLL dependencies are placed in the same directory
  • Add the DLL directory to system PATH
  • Ensure extern "C" is added to the C++ header file to prevent name mangling
  • Using keyword EXPORTED_FUNCTION in the C++ header file
  • Setting the current working directory in Python to that of the DLL file directory prior to loading DLLs

without any success...

I've also compiled my own SEPARATE 64-bit C++ DLL file (completely independent from the aa_Functions.dll file previously mentioned) in Microsoft Visual Studio 2010 in both Debug and Release modes, with test functions included inside.

This test DLL is able to be loaded, and its functions called successfully, per the following:

__path_test_dll__ = r'C:\user\dllTest\x64\Release\dllTest.dll' # test DLL file path
test_dll = CDLL(__path_test_dll__) # load test DLL
test_dll.fn4.restype = c_void_p # set test DLL function return type
test_dll.fn4.argtypes = [c_double, c_double, c_double, c_char_p, c_bool] # set test DLL function argument types
test_dll.fn4(10, 12, 1, b"Pay", True) # make test DLL function call

With the expected console output:

Min_P * Rate == $10
Pay: 1 Min_P: 10 Max_P: 12 Iteration #: 1
Min_P * Rate == $11
Pay: 0 Min_P: 11 Max_P: 12 Iteration #: 2
Min_P * Rate == $12
Pay: 1 Min_P: 12 Max_P: 12 Iteration #: 3

Process finished with exit code 0

I've spent the last several days on this issue and a point in the right direction would be tremendously appreciated.

ilbi
  • 51
  • 5
  • The ABI of C++ object files/libraries is usually not stable across compiler vendors/versions. Are you sure the dll was compiled with the exact same compiler as the program you are running that tries to load the dll? If not, you *may* have some fun trouble. – Jesper Juhl May 23 '18 at 21:22
  • Changing the header file to have `extern "C"`, `EXPORTED_FUNCTION`, etc., without recompiling the DLL won't work. If those were missing, likely the functions are exported as C++-mangled names instead of C names. Use `dumpbin /EXPORTS your.dll` to see what the names look like. Also `LOAD_LIBRARY_AS_DATAFILE` doesn't load the function table from the DLL, IIRC. – Mark Tolonen May 23 '18 at 21:54
  • The fact that it can be loaded by *pywin32* but not by *ctypes* doesn't make sense. You're definitely doing smth wrong (maybe not using the same path)? So what is your *Python* (also *ctypes*, and *pywin32*) version? What tools are used to compile the *.dll* by the other person? The 2nd case it's normal not to find the symbol, based on how you load the dll. So with the test dll which loads the dll (everything compiled by you) everything works? What about with test dll which loads the external compiled dll? – CristiFati May 24 '18 at 10:36
  • @MarkTolonen The DLL was indeed compiled by the other person with `extern "C"`, `EXPORTED_FUNCTION` already in the header file. Using `dumpbin /exports aa_Function.dll` I could see the function names, which match the names shown in Dependency Walker. I am making function calls in Python using the same function names in Python - unsuccessfully. Your point about `LOAD_LIBRARY_AS_DATAFILE` not loading the DLL function table is interesting. Couldn't find anything in the documentation about this method of loading. This would make sense as the DLLS are loaded, but functions can't be called. – ilbi May 24 '18 at 15:20
  • I'm using Python 3.6.4 64-bit, (built using MSC v.1900 on win32, which is Visual Studio 2013 ver 14.0) through module Ctypes to try and load DLLs compiled using Visual Studio 2013. The person who compiled the DLLs said this difference in compiler versions should not cause any conflicts... Unsure what next steps should be... – ilbi May 24 '18 at 15:28
  • @CristiFati _Python_ version: 3.6.4 _Ctypes_ version: 1.1.0 _pywin32_ version: 222. From my understanding, the other person used Visual Studio 2013 to compile the _.dll_. The test _.dll_ I wrote from scratch and loaded in Python works successfully, I can also call its functions. The _.dll_ compiled by the other person does not load through `ctypes.CDLL(dll_path)`. However, this _.dll_ will load using the `LOAD_LIBRARY_AS_DATAFILE`, but as Mark suggested, loading DLL this way may not load its function table, which may explain why functions cannot be called given this DLL load method. – ilbi May 24 '18 at 15:35
  • Read about `LOAD_LIBRARY_AS_DATAFILE` [here](https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx). It simply maps the DLL into memory and does no other preparation like looking for other dependencies or preparing the functions to be called. If method 1 doesn't work, it looks like a dependency is missing. – Mark Tolonen May 24 '18 at 15:56
  • Update: may _potentially_ be a MATLAB (dependency) issue. Prior to any MATLAB license configurations, the dependency _libmx.dll_ could not be loaded in Python via `ctypes.CDLL()`. However, after initial license configurations, the _libmx.dll_ is able to be loaded. But given that MATLAB is not 100% configured yet, the _aa_Functions.dll_ still cannot be loaded. Will provide further updates as they're received and verified. – ilbi May 25 '18 at 14:18

1 Answers1

0

The solution to my specific problem was:

  1. Ensure a proper installation of MATLAB
  2. Ensure the dependent MATLAB DLL (libmx.dll) directory is added to PATH and can be loaded using ctypes PRIOR to loading the actual DLL

In hindsight, this was a very straightforward issue - missing DLL dependency. But it can be a challenge to find the SPECIFIC missing DLL.

ilbi
  • 51
  • 5