IDL-Python bridge fails due to SIP on OSX El Capitan

Question

I'm attempting to utilize the new IDL-Python bridge in IDL 8.5.1 on OSX El Capitan 10.11.5. I have used this feature on Windows since it was launched, which works beautifully, but it simply doesn't work on OSX.

I have installed Anaconda Python 3.4.1 64bit to test.

After setting up the environment as described here, which involves setting the DYLD_LIBRARY_PATH environment variable.

I am able to call IDL from Python, but not the other way around. For example, if I run the following command(from an example here), the following error is observed:

IDL> ran = Python.Import('numpy.random')
% DLM_LOAD: Error loading sharable executable.
            Symbol: IDL_Load, File = /Applications/exelis/idl85/bin/bin.darwin.x86_64/idl_python34.so
            dlopen(/Applications/exelis/idl85/bin/bin.darwin.x86_64/idl_python34.so, 1): Library not loaded: libpython3.4m.dylib
              Referenced from: /Applications/exelis/idl85/bin/bin.darwin.x86_64/idl_python34.so
              Reason: image not found
% Execution halted at: $MAIN$

After working with their tech support (and I have to say, they have been very helpful and their help has been of a high quality) we determined the problem is caused by the new System Integrity Protection (SIP) feature in OSX. This problem has been described on stack before.

Their suggestion was to disable the SIP, but I neither have the permission from my employer to do so, nor the willingness to turn off a system security feature just to get one piece of commercial software to work. Therefore I am stuck.

Essentially, the DYLD_LIBRARY_PATH variable is simply not passed on to the IDL process. When launching IDL, the /Applications/exelis/idl85/bin/idl shell script is executed, and at that point the variable is stripped of any modifications that I have made, and the idl script simply is unaware of my changes.

My question then, is this: Has anyone else found an alternative way (other than disabling the SIP) to getting the IDL-Python bridge to work? Are there any alternatives here? Or will this simply not work until the good folks at Harris Geospatial find another way to implement this feature?

This is a serious design problem in my opinion, if it requires modifying the system at the root level just to run their software.

Answer 1

Okay, we are currently working on ways to more easily get the IDL-Python bridge to work on El Capitan. In the meantime, here are the steps you can take to get it to work. First, make sure you have your paths set up correctly. For example, in your .login file (not .cshrc):

setenv PATH /Users/username/anaconda/bin:${PATH}
setenv PYTHONHOME /Users/username/anaconda
setenv PYTHONPATH /Users/username/Applications/exelis/idl/bin/bin.darwin.x86_64
setenv PYTHONPATH ${PYTHONPATH}:/Users/username/Applications/exelis/idl85/lib/bridges

Then, for the IDL-to-Python bridge, run the following commands:

cd Applications/exelis/idl85/bin/bin.darwin.x86_64/
sudo install_name_tool -change libpython3.4m.dylib /Users/username/anaconda/lib/libpython3.4m.dylib idl_python34.so

You can use this command to verify:

otool -L idl_python34.so

For the Python-to-IDL bridge (some of these are duplicated from above):

cd Applications/exelis/idl85/bin/bin.darwin.x86_64/
sudo install_name_tool -change libidl.8.5.dylib @loader_path/libidl.8.5.dylib pythonidl34.so
sudo install_name_tool -change libidl_ips.8.5.dylib @loader_path/libidl_ips.8.5.dylib pythonidl34.so
sudo install_name_tool -change libpython3.4m.dylib /Users/username/anaconda/lib/libpython3.4m.dylib pythonidl34.so

At this point, DYLD_LIBRARY_PATH is still needed in the .login to specify IDL’s bin directory. To eliminate that, the following updates can be made to tell various libraries where to find their dependencies:

sudo install_name_tool -change libMesaGL6_2.dylib @loader_path/libMesaGL6_2.dylib libidl.8.5.dylib
sudo install_name_tool -change libMesaGLU6_2.dylib @loader_path/libMesaGLU6_2.dylib libidl.8.5.dylib
sudo install_name_tool -change libOSMesa6_2.dylib @loader_path/libOSMesa6_2.dylib libidl.8.5.dylib
sudo install_name_tool -change libXm.3.0.2.dylib @loader_path/libXm.3.0.2.dylib libidl.8.5.dylib
sudo install_name_tool -change libMesaGL6_2.dylib @loader_path/libMesaGL6_2.dylib libMesaGLU6_2.dylib
sudo install_name_tool -change libMesaGL6_2.dylib @loader_path/libMesaGL6_2.dylib libOSMesa6_2.dylib
sudo install_name_tool -change libidl.8.5.dylib @loader_path/libidl.8.5.dylib libidl_ips.8.5.dylib

Hope this helps!