How to call MATLAB executable for Python on cluster?

Question

I am using a python-matlab-bridge that calls MATLAB from python by starting it on a ZMQ socket. On my own computer, I hand the bridge the location of the executable (in this case MATLAB 2014B):

executable='/Applications/MATLAB_R2014b.app/bin/matlab'

and everything works as required and the printed statement is:

Starting MATLAB on ZMQ socket ipc:///tmp/pymatbridge-49ce56ed-f5b4-43c4-8d53-8ae0cd30136d

Now I want to do the same on a cluster. Through module avail I find there are two MATLAB versions (2015a and 2016b) available and located at the following path:

/opt/ud/LOCAL/etc/modulefiles/matlab.

When I now call MATLAB using:

executable='/opt/ud/LOCAL/etc/modulefiles/matlab/MATLAB_R2015a.app/bin/matlab'

the error:

 Starting MATLAB on ZMQ socket ipc:///tmp/pymatbridge-95775445-359d-441f-803a-7193eedbf215    
Send 'exit' command to kill the server
............................................................MATLAB session timed out after 60 seconds

is returned. It cannot find the MATLAB executable. How to proceed?

Can you start MATLAB manually with that path? Are those Linux servers in the cluster? — hbaderts, Mar 15 '17 at 10:38

score 1 · Answer 1 · edited May 12 '17 at 16:02

Step 0:

Check whether the code-segment responsible for "remote" launch of MATLAB goes well - for details about params, check pymatbridge.py:

    ### class _Session( object ): _____________________________
    ...

    def _run_server(self):
        code = self._preamble_code()
        code.extend([
            "matlabserver('%s')" % self.socket_addr
        ])
        command = '%s %s %s "%s"' % ( self.executable,
                                      self.startup_options,
                                      self._execute_flag(),
                                      ','.join( code )
                                      )
        subprocess.Popen( command,
                          shell  = True,
                          stdin  = subprocess.PIPE,
                          stdout = subprocess.PIPE
                          )

If such invocation works on any Cluster-Node you will try to harness in a massively distributed MATLAB scenario, the problem goes into ZeroMQ messaging part, if not, the remote server simply cannot launch a _Session, while the messaging ( and all it's { ipc:// | tcp:// | pgm:// | epgm:// | vmci:// } transport-classes available for a massively-distributed computing ) is not to be blamed.

Step 1:

If Step 0 ( a subprocess.Popen(...) invocation ) works well on all remote nodes, check all the ZeroMQ / pymatbridge pre-requisites ( dynamic loader settings - lines added to all remote cluster nodes' .bash_profile ( or similar file for respective shell ).

Yes, a tedious task to prove all remote nodes on the cluster meet these, but without checking these, there is no warranty a remote node can operate the remote MATLAB process connected via otherwise smart and powerful tool, like the pymatbridge is.

You were right in the fact that the problem was the remote launch of MATLAB. — H. Vabri, Mar 20 '17 at 15:07

score 0 · Accepted Answer · answered Mar 20 '17 at 15:06

In the end I found a method that works that is very simple. In the .pbs script I load the MATLAB module using

module load matlab/2015a

Then in the Python script the command

mlab = Matlab()

automatically detects the correct MATLAB path. I thus do not need to specify the path using the optional executable argument.

How to call MATLAB executable for Python on cluster?

2 Answers2

Step 0:

Step 1:

Linked