Submitting a matlab serial job to HPC

Question

I want to submit serial matlab script on HPC server using:

Code:

#!/bin/bash 
#$ -N matlabjob 
#$ -q all.q 
#$ -pe mpi 1 
/opt/matlab/bin/matlab -nodesktop -nosplash -r "run /home/abhishekb/Matlab/new.m;quit" > out.txt

Error:

License checkout failed.
License Manager Error -15
MATLAB is unable to connect to the license server. 
Check that the license manager has been started ,and that the MATLAB client machine can communicate
with the license server.
Troubleshoot this issue by visiting: 
http://www.mathworks.com/support/lme/R2013b/15
Diagnostic Information:
Feature: MATLAB 
License path: /home/abhishekb/.matlab/R2013b_licenses:/opt/matlab/licenses/license.dat:/opt/matlab/licenses/networ
k.lic 
Licensing error: -15,570. System Error: 115

But this doesn't work as IT guy told me " as we don't have distributed computing license so parallel jobs of matlab can't run".

So I tried to run it background as:

#!/bin/bash
nohup /opt/matlab/bin/matlab -nodesktop -nosplash -r "run /home/abhishekb/Matlab/New_edited3.m;quit" > output.log </dev/null &
echo $! > save_pid.txt

But it seems to run unbearably slow. Even reading .csv files takes time. I am comparing to the run on my pc based just on the output intervals.

Can someone help me how to submit a serial job which doesn't require "matlab's distributed computing license". If this makes sense.

I use qsub for submitting jobs.

EDIT:

>>  mcc -m automate.m
Undefined function 'mcc' for input arguments of type 'char'.
>> ver
------------------------------------------------------------------------------**strong text**----------------------
MATLAB Version: 8.2.0.701 (R2013b)
MATLAB License Number: _______
Operating System: Linux 2.6.32-431.11.2.el6.x86_64 #1 SMP Tue Mar 25 19:59:55 UTC 2014 x86_64
Java Version: Java 1.7.0_11-b21 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
----------------------------------------------------------------------------------------------------
MATLAB                                                Version 8.2        (R2013b)
Simulink                                              Version 8.2        (R2013b)
Bioinformatics Toolbox                                Version 4.3.1      (R2013b)
Communications System Toolbox                         Version 5.5        (R2013b)
Computer Vision System Toolbox                        Version 5.3        (R2013b)
Control System Toolbox                                Version 9.6        (R2013b)
Curve Fitting Toolbox                                 Version 3.4        (R2013b)
DSP System Toolbox                                    Version 8.5        (R2013b)
Fixed-Point Designer                                  Version 4.1        (R2013b)
Global Optimization Toolbox                           Version 3.2.4      (R2013b)
Image Processing Toolbox                              Version 8.3        (R2013b)
Neural Network Toolbox                                Version 8.1        (R2013b)
Optimization Toolbox                                  Version 6.4        (R2013b)
Parallel Computing Toolbox                            Version 6.3        (R2013b)
Partial Differential Equation Toolbox                 Version 1.3        (R2013b)
RF Toolbox                                            Version 2.13       (R2013b)
Signal Processing Toolbox                             Version 6.20       (R2013b)
SimBiology                                            Version 4.3.1      (R2013b)
Statistics Toolbox                                    Version 8.3        (R2013b)
Symbolic Math Toolbox                                 Version 5.11       (R2013b)
Wavelet Toolbox                                       Version 4.12       (R2013b)
>>

Hum. Not very good at this, but, if you submit a job to an HPC without the distributed computing license, what you are doing is just assignin a single core of that HPC to run your Matlab code. So the top speed youll get is as fast as that single core is, which is probably slower than your desktop. — Ander Biguri, Jul 28 '15 at 14:20
Not sure how calling your m script a different way should fix licensing errors. If you don't have the licence, you cant use the functions from the distributed computing toolbox. — Daniel, Jul 28 '15 at 16:32
I am quite surprised you got this upvoted. If you dont pay for the license, you cant do what the license provides, it is obvious. — Ander Biguri, Nov 02 '15 at 10:49
Where is the documentation for your HPC? I've never seen the `-pe` option and can't find anything about it anywhere. Also why can you not ask your "IT guy"? — IKavanagh, Nov 03 '15 at 17:16
Maybe instead of trying to get your code to run in a HPC, all this effort could have been invested in trying to make your code faster through vectorization... just saying. What you could do is show us the code you're running, and we can provide tips on how to make it faster instead of trying to rely on a toolbox that you don't have a license to. [Have you ever heard of the XY Problem?](http://xyproblem.info/) — rayryeng, Nov 05 '15 at 17:01

score 3 · Answer 1 · answered Nov 05 '15 at 20:30

tl;dr: Get your "IT guy" to fix this. We can't.

I'm going to answer this possibly against my better judgement.

You are submitting the job "serially" because the #$ -pe mpi 1 option specifies you want only 1 node. The reason your Matlab code doesn't run is because of an issue with the configuration of the Matlab license server on your HPC. There is no workaround for this except to fix the configuration. Mathworks provide a starting point for you to debug this. However, you most likely won't be able to and will need your "IT guy" or those who manage your HPC to.

To add to this when you submit your job in the background you also submit it "serially". It ran extremely slow on this because it was running on 1 core which was probably also processing requests from other users as this is what's called the head (or login) node. In contrast when you execute this code on your own machine it most likely leverages multiple cores as opposed to the single core available on the head node.

Thanks to @Andras Deak for answering the question in my comment, which enabled me to reason out an answer with his help.

score 1 · Answer 2 · answered Jul 30 '15 at 16:26

One of the ways to get around this will be to crate an executable by compiling your Matlab code using mcc

Example:

We need to convert the script to a function, let us assume that new.m does a simple hello world

for iter = 1 : 10
fprintf('Hello, world %d \n', iter);
end

to a function

function hello
for iter = 1 : 10
fprintf('Hello, world %d \n', iter);
end

and then use mcc to produce executables.

mcc -m new.m

This produces files : new, new_main.c, new_mcc_component_data.c, new.prj and the exececutable is new.

You can now run new just like any other executable in Linux.

If you are using a scheduler to submit your job, you can put the executable new as the command to be invoked in the submission script.

Submitting a matlab serial job to HPC

2 Answers2