Loading Compiled Matlab Shared Library in Python Using Ctypes

Question

I am trying to do Incomplete Cholesky Decomposition in Python, but no direct Python package I can find.

Since the most available codes I can find online are written in Matlab, I want to take a detour by

• compiling the matlab code to a shared library (I am using Mac OS and MATLAB_R2014a, so it should produce .dylib file)
• load library in Python by using Ctypes

The following lists the detailed steps:

0. Download Matlab Source Code

The code can be downloaded from F. Bach's webpage link to zip file, which contains the following files:

panc:csi-1.0 panc25$ ls
center.m            csi.dll             csi.mexglx          csi_gaussian.dll    csi_gaussian.mexglx readme.txt
csi.c               csi.m               csi_gaussian.c      csi_gaussian.m      demo_csi.m          sqdist.m

1. Compiling the matlab code to a shared library

Then by following this post, I run the command:

mcc -v -W cpplib:libcsi -T link:lib csi

After around a minute, the terminal prints MEX completed successfully and in my folder there are

panc:csi-1.0 panc25$ ls
center.m             csi.m                csi_gaussian.dll     demo_csi.m           libcsi.exports       readme.txt
csi.c                csi.mexglx           csi_gaussian.m       libcsi.cpp           libcsi.h             sqdist.m
csi.dll              csi_gaussian.c       csi_gaussian.mexglx  libcsi.dylib         mccExcludedFiles.log

where libcsi.dylib is the shared library I want.

2. Loading library in Python

Then I open IPython and try to load the library:

In [1]: import ctypes

In [2]: ctypes.C
ctypes.CDLL       ctypes.CFUNCTYPE

In [2]: ctypes.CDLL('libcsi.dylib')
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-2-b6d0c1a91651> in <module>()
----> 1 ctypes.CDLL('libcsi.dylib')

/Users/panc25/anaconda/lib/python2.7/ctypes/__init__.pyc in __init__(self, name, mode, handle, use_errno, use_last_error)
    363
    364         if handle is None:
--> 365             self._handle = _dlopen(self._name, mode)
    366         else:
    367             self._handle = handle

OSError: dlopen(libcsi.dylib, 6): Library not loaded: @rpath/libmwmclmcrrt.8.3.dylib
  Referenced from: /Users/panc25/Downloads/csi-1.0/libcsi.dylib
  Reason: image not found

This problem persists even after I replace file name in ctypes.CDLL('libcsi.dylib') with the full path.

So I am confused. The shared library is there, but why Python says "image not found"?

BTW

SInce the source code also provide C implementation through mex.h, I also tried to first create a .mex file, then compile the .mex to a shared library as follows:

panc:csi-1.0 panc25$ mex csi.c

which created the csi.mexmaci64 file. Then according to this link, I called:

panc:csi-1.0 panc25$ mcc -B csharedlib:csi2 csi.mexmaci64

which produced csi2.dylib file.

But when I tried to load it in Python, I had the same error.

Could anyone let me know what is wrong?

Answer 1

I would avoid Matlab altogether, and instead use the Incomplete Cholesky Decomposition available in PyMC2:

from pymc.gp.incomplete_chol import ichol_full

The f2py wrapped Fortran code, that was actually adapted from a MEX file, can be found here. So you could use this independently of PyMC2 if need be.

If you are interested, you could also propose to add this function to scipy (see this githib issue ).