Matlab Engine Python - OSx Anaconda Segfault or DYLD_LIBRARY_PATH error with iPython

Question

After installing the matlab python package in terminal using:

cd "matlabroot\extern\engines\python"
python setup.py install

And trying to run it, I get a segfault:

:~$ python
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, May 28 2015, 17:04:42) 
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import matlab.engine
Segmentation fault: 11

However, I can get around this by setting DYLD_LIBRARY_PATH after which matlab.engine works:

:~$ export DYLD_LIBRARY_PATH=/System/Library/Frameworks/Python.framework/Versions/Current/lib:$DYLD_LIBRARY_PATH
:~$ python
Python 2.7.6 (default, Sep  9 2014, 15:04:36) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matlab.engine
>>> eng = matlab.engine.start_matlab()
>>> exit()

However, when I try to launch iPython afterwards I get this error:

Traceback (most recent call last):
  File "//anaconda/bin/ipython", line 4, in <module>
    from IPython import start_ipython
  File "//anaconda/lib/python2.7/site-packages/IPython/__init__.py", line 45, in <module>
    from .config.loader import Config
  File "//anaconda/lib/python2.7/site-packages/IPython/config/__init__.py", line 6, in <module>
    from .application import *
  File "//anaconda/lib/python2.7/site-packages/IPython/config/application.py", line 19, in <module>
    from IPython.config.configurable import SingletonConfigurable
  File "//anaconda/lib/python2.7/site-packages/IPython/config/configurable.py", line 12, in <module>
    from .loader import Config, LazyConfigValue
  File "//anaconda/lib/python2.7/site-packages/IPython/config/loader.py", line 16, in <module>
    from IPython.utils.path import filefind, get_ipython_dir
  File "//anaconda/lib/python2.7/site-packages/IPython/utils/path.py", line 14, in <module>
    import tempfile
  File "//anaconda/lib/python2.7/tempfile.py", line 32, in <module>
    import io as _io
  File "//anaconda/lib/python2.7/io.py", line 51, in <module>
    import _io
ImportError: dlopen(//anaconda/lib/python2.7/lib-dynload/_io.so, 2): Symbol not found: __PyErr_ReplaceException
  Referenced from: //anaconda/lib/python2.7/lib-dynload/_io.so
  Expected in: dynamic lookup

As you can see the python versions are different. I think this is a conflict between my system Python and Anaconda but I'm not sure how to fix it, any help much appreciated. Thanks.

Answer 1

You need to install libraries using the conda command line install utility, not python. conda help search and conda help install should get you going.

Answer 2

This is not a direct solution, but I was stuck with the same problem and used this workaround. And I should say that I did finally make matlab.engine run in my Python environment, but that it just plain sucked. It couldn't run scripts with embedded scripts in other folders, and I also experienced that it couldn't find some build-in functions. The following is implemented for a Unix machine, but it would not take a lot of modification to make it work in Windows, I believe.

Here's what I did:

1. Wrote a main Matlab script that could do everything I needed from Matlab.
2. Ran that script through a subprocess in Python.

In my case I needed do a series of matrix operations for matrix X and return matrix S. The matrix operations required the use of a particular Matlab function. My first idea was to open a Matlab-session with matlab.engine and then manage the matrix operations in Python only calling the Matlab function when needed. Instead (as the bullets state) I wrote a Matlab function ComputeSimilarityMat that takes X, does all necessary operations, and returns S. Then I basically just ran that Matlab function from Python using a subprocess.

This is what the Python script, that manages the Matlab script through a subprocess, looks like:

import subprocess
import numpy as np

def run_matlab(X):
    """Produce trait-similarity matrix S from a trait-space matrix X

    Parameters
    ----------
    X : numpy.ndarray

    Returns
    -------
    S : numpy.ndarray
    """

    # Dump input in .csv, so Matlab can load it
    np.savetxt('X.csv', X, delimiter=",")

    # Code executed in Matlab. Important to end with quit;
    matlab_code = "X=csvread('X.csv');" \
                  "S=ComputeSimilarityMat(X);" \
                  "csvwrite('S.csv',S);" \
                  "quit;"

    # -nosplash suppresses the splash window on startup
    matlab_call = ["matlab", "-nodesktop", "-nosplash", "-r", matlab_code]
    subprocess.call(matlab_call)

    # Load the .csv file that Matlab dumps after finishing
    S = np.genfromtxt('X.csv', delimiter=",")

    return S

I have to say, I'm sure there's a nicer way to pass an object to Matlab then to have to save and load like this, but it worked for me. Still hoping for an improvement of matlab.engine though, but until then I'm going with this approach.

Note: To run Matlab from command-line/subprocess you need to add the Matlab bin folder to your PATH variable like this:

export PATH="/Applications/MATLAB_R2015a.app/bin:$PATH"

I just put this in my .profile file.

Answer 3

In my case adding things to LD_LIBRARY_PATH (Ubuntu's version of DYLD_LIBRARY_PATH) only made things worse. Instead I had to make sure it did not refer to the Python installation, and had to add a symbolic link from /usr/lib instead. See https://stackoverflow.com/a/45161288/2524427