1

I am under windows. The fortran code in mkl_example.f :

    subroutine matmultmkl(M1, M2, M3, M, N, K) bind(c, name='matmultmkl')
        !DEC$ ATTRIBUTES DLLEXPORT :: matmultmkl
        use iso_c_binding, only: c_float, c_int
    
        integer(c_int),intent(in) :: M, N, K
        real(c_float), intent(in) :: M1(M, N), M2(N, K)
        real(c_float), intent(out):: M3(M, K)

        CALL DGEMM('N','N',M,K,N,1.,M1,M,M2,N,0.,M3,M)
    
    end subroutine

I compile it with command line (in a cmd windows where I ran compilervars.bat before) using the following batch file :

@Echo off

setlocal ENABLEDELAYEDEXPANSION
SET "IFORT_INITIAL_FLAGS=-c -fpp"
SET "IFORT_OPTIMIZATION_FLAGS=/O3"

ifort %IFORT_OPTIMIZATION_FLAGS% %IFORT_INITIAL_FLAGS% /I"C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2020.4.311\windows\mkl\include" -o mkl_example.obj mkl_example.f
ifort -dll -o mylib.dll mkl_example.obj /link /LIBPATH:"C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2020.4.311\windows\mkl\lib\intel64_win" mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib

Then I run the follow python script mkl_example.py :

from ctypes import *
import time

import os
os.add_dll_directory("C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020.4.311/windows/mkl/lib/intel64_win")
os.add_dll_directory("C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020.4.311/windows/compiler/lib/intel64_win")

import numpy as np

mylib = CDLL('./mylib.dll')

mylib.matmultmkl.argtypes = [ POINTER(c_float), 
                                POINTER(c_float), 
                                POINTER(c_float),
                                POINTER(c_int),
                                POINTER(c_int),
                                POINTER(c_int) ]
mylib.matmultmkl.restype = None
M=1000
N=1000
K=1000

a = np.empty((M,N), dtype=c_float)
b = np.empty((N,K), dtype=c_float)
c = np.empty((M,K), dtype=c_float)

a[:] = np.random.rand(M,N)
b[:] = np.random.rand(N,K)

# Fortran mkl call
start = time.time()
mylib.matmultmkl( a.ctypes.data_as(POINTER(c_float)), 
                    b.ctypes.data_as(POINTER(c_float)), 
                    c.ctypes.data_as(POINTER(c_float)), 
                    c_int(M), c_int(N), c_int(K) )
stop = time.time()
print(f"Fortran mkl \t {stop - start}s")

The output is :

Traceback (most recent call last):
  File "path\to\mkl_example.py", line 14, in <module>
    mylib = CDLL('./mylib.dll')
  File "C:\PYTHON\3\3.8.2_64bits\lib\ctypes\__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: path\to\mylib.dll' (or one of its dependencies). Try using 
the full path with constructor syntax.

If y remove all the mkl stuff and code a function myself, the mylib.dll is found at runtime by the python. Hence in the case where I link to the mkl, the message doesn't mean that even if mylib.dll is at the right place it is not found at runtime for some weird reason, but thet indeed one of it's dependencies is not found.

I don't see what I can do except finding all dependencies at putting them in the same folder next to the fortran dll and the python file. But if this is the only way, maybe I prefer quitting definitely.

Ian Bush
  • 6,996
  • 1
  • 21
  • 27
Olórin
  • 3,367
  • 2
  • 22
  • 42

1 Answers1

0

The paths being added to the DLL search path in the python script are for directories that contain the static and import libraries for the compiler and MKL, not the runtime DLLs.

(Installation of the Intel compiler typically installs the compiler runtime in a Common files directory, and adds that to the system search path - so the compiler runtime DLLs are likely being found by that route - but this does not apply to the MKL runtime DLLs.)

Use the correct directory in the os.add_dll_directory call. It will be something like (you need to check your installation) C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_xxxx\windows\redist\intel64_win\mkl. Version agnostic symlinked directories may also be more suitable, depending on your needs.

(Deployment to another machine without the compiler installed will require a strategy around how you deploy dependencies, but this is a much larger topic.)

IanH
  • 21,026
  • 2
  • 37
  • 59
  • It indeed "worked" when I added with `os.add_dll_directory` all directories containing the `.lib`'s from the second compiling line from my `.bat` file. Then some strange thing happpened : nothing. Then in the python I change the size from 1000 to 100, and then it worked. I suspect that `dgemm` triggers a segmentation fauil for a product of 1000*1000 square matrices, but not for 100*100 matrices. This is really strange. – Olórin Feb 25 '21 at 23:23
  • I created a question regarding my above comment question : https://stackoverflow.com/questions/66387002/segmentation-fault-when-using-fortran-intels-mkl-dgemm-with-python-ctypes-alrea – Olórin Feb 26 '21 at 13:50