0

I'm running the MATLAB using the Python's subprocess like this. The main.py is as follows:

process = subprocess.Popen("bash run.sh", stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
stdout = process.communicate()[0]

and the run.sh is like this:

export PATH=...
/usr/local/MATLAB/R2017b/bin/matlab -r "run('mainscript.m');exit;"

and in mainscript.m, the main things done there.

The problem is, when there are some errors occurred in running the mainscript.m, the process seems being stuck. And the MATLAB process would not exit normally. It seems the MATLAB session is still running without being exited, and thus the main.py is still running without exit, which seems being stuck. But actually the main.py needs to exit or gives some alerts.

So my question is, is there any way to exit the MATLAB session when errors occurred in /usr/local/MATLAB/R2017b/bin/matlab -r "run('mainscript.m');exit;?

Looking forward to your advice. Thanks.

UPDATED:

After reading the kind comments and answer, I supply more details for clarity on this question.

The main.py is:

import subprocess
import time
from threading import Thread


def main(param1):
    process = subprocess.Popen('bash run.sh', stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
    start = time.time()
    stdout = process.communicate()[0]
    elapsed = time.time() - start
    print('log: {}, {}, {}'.format(param1, stdout, elapsed))


if __name__=='__main__':
    _param = {"param1":"param1"}
    thread = Thread(target=main, kwargs=_param)
    thread.start()

The run.sh is:

/usr/local/MATLAB/R2017b/bin/matlab -r "run('matlabscript.m');exit;"

The matlabscript.m is (xxxx is not a valid function/script, which leads to errors.):

xxxx;

When running the python main.py, the outputs of ps -u are:

ubuntu    4901  0.0  0.0 218624  5392 pts/4    Sl+  08:10   0:00 python main.py
ubuntu    4903  0.0  0.0 113280  1192 pts/4    S+   08:10   0:00 bash run.sh
ubuntu    4904  9.0  3.1 5702484 512812 pts/4  Sl+  08:10   0:06 /usr/local/MATLAB/R2017b/bin/glnxa64/MATLAB -r run('matlabscript.m');exit; -prefersoftwareopengl
ubuntu    5025  0.0  0.0 115544  2004 pts/3    Ss   08:11   0:00 -bash
ubuntu    5044  0.0  0.0 155436  1852 pts/3    R+   08:11   0:00 ps -u

And the python main.py is being stuck there and does not exit normaly. So I kill 4901, then ps -u, it shows:

ubuntu    4903  0.0  0.0 113280  1192 pts/4    S    08:10   0:00 bash run.sh
ubuntu    4904  4.7  3.1 5702484 512812 pts/4  Sl   08:10   0:06 /usr/local/MATLAB/R2017b/bin/glnxa64/MATLAB -r run('matlabscript.m');exit; -prefersoftwareopengl
ubuntu    5025  0.0  0.0 115544  2052 pts/3    Ss   08:11   0:00 -bash
ubuntu    5047  0.0  0.0 155436  1852 pts/3    R+   08:12   0:00 ps -u

This means the subprocess of MATLAB is still running without exit.

But, the following is OK. The main_with_try_catch.py is:

import subprocess
import time
from threading import Thread


def main(param1):
    process = subprocess.Popen('bash run_with_try_catch.sh', stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
    start = time.time()
    stdout = process.communicate()[0]
    elapsed = time.time() - start
    print('log: {}, {}, {}'.format(param1, stdout, elapsed))


if __name__=='__main__':
    _param = {"param1":"param1"}
    thread = Thread(target=main, kwargs=_param)
    thread.start()

The run_with_try_catch.sh is:

/usr/local/MATLAB/R2017b/bin/matlab -r "try,run('matlabscript.m'),catch,fprintf('error occured'),end;exit;"

Then, run the python main_with_try_catch.py, it shows:

[ubuntu@localhost ~]$ python main_with_try_catch.py
log: param1, MATLAB is selecting SOFTWARE OPENGL rendering.

                                                                              < M A T L A B (R) >
                                                                    Copyright 1984-2017 The MathWorks, Inc.
                                                                     R2017b (9.3.0.713579) 64-bit (glnxa64)
                                                                               September 14, 2017


To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

error occured, 7.47159695625
[ubuntu@localhost ~]$

This means the MATLAB is exited normally. And check the ps -u, there is no leftover processes.

mining
  • 3,557
  • 5
  • 39
  • 66
  • In real applications, you should not error. So if there is an error expected, then you try-catch it, then handle it there, by e.g. exiting. If you are just developing the script, then its highly suggested you develop it in MATLAB and make sure there are no errors before calling it from python. I can not think any other alternative than wrapping the entire matlab script in a try catch, but that sucks – Ander Biguri Oct 01 '21 at 08:59
  • @AnderBiguri, thank you very much for your kind suggestion! You remind me of the try-catch-end block in MATLAB. And using the try-catch-end block, my question seems solved by `/usr/local/MATLAB/R2017b/bin/matlab -r "try,run('test.m');catch,fprintf('errors'),end;exit;"`. Thank you! – mining Oct 01 '21 at 09:57
  • But that is a bad solution, because it makes your code much slower – Ander Biguri Oct 01 '21 at 10:04
  • @AnderBiguri, yeah, the try-catch block would slow down the code. Currently, due to time limit, I'm trying to get all things done. Then I will consider how to improve the efficiency. Thank you very much for your comments and kind suggestions ! – mining Oct 01 '21 at 10:39
  • By stuck do you mean defunct state (D)? You can check this by ps -ef | grep defunct If so, you can kill -9 [PID] [PPID] to terminate your program. In my experience using Matlab on Ubuntu, it can easily go on defunct state and won't fix the error by itself. – GuarneerFPS Oct 01 '21 at 11:12
  • @GuarneerFPS, thank you very much for your kind comment. I've updated the post for clarity, please check that. Thanks. – mining Oct 01 '21 at 12:25

2 Answers2

3

Do not use -r in MATLAB’s startup to run a script. Use the -batch option instead.

The -r option says “start MATLAB and run this script, stay at the command prompt when it’s done”. The script is meant to initialize the environment. This is why your script needs to end with exit and not error out if you want to run it non-interactively.

The -batch option says “run this script and then exit MATLAB with a success/failure status”. It is meant to execute a script written in the MATLAB language non-interactively. It does not show the splash screen nor load the GUI, producing output through stdout and stderr. See the docs. This option exists since R2019a.

Cris Luengo
  • 55,762
  • 10
  • 62
  • 120
  • Thank you very much for your kind answer and suggestions! Actually the program is running on a CentOS server without GUI, so I think your solution is exactly what I need. Currently I only have the R2017b version of MATLAB. Thank you! – mining Oct 01 '21 at 23:19
1

Run the subprocess in a thread, to terminate it after a given timeout if not exited normally. Find examples here: https://stackoverflow.com/a/4825933/4620675

To close process you can use just a normal kill in subprocess: kill -TERM $(pgrep -f matlab)

Or using method:

    import os
    import signal
    def find_process_and_close(process_name):
        output = subprocess.getoutput("ps -A -o pid= -o cmd=")
        for line in output.splitlines():
            if process_name in line:
                pid = int(line.split()[0])
                os.kill(pid, signal.SIGTERM)
                print("Terminate process {}".format(line))
                return True
        return False

Another method with psutil package:

    import psutil
    def find_process_and_close_with_psutil(process_name):
        for proc in psutil.process_iter(['pid', 'name', 'cmdline']):
            if proc.info['cmdline'] and process_name in proc.info['cmdline']:
                print("Terminate process {}".format(proc.info))
                proc.kill()
                return True
        return False

EDIT After reading comment. You can read output from script continuously and if found interesting string, e.g. 'error' message, just terminate running process (with all child processes):

    proc = subprocess.Popen(["bash run.sh"],
                            preexec_fn=os.setsid, # to kill all spawned processes
                            stdout=subprocess.PIPE,
                            stderr=subprocess.STDOUT)
    pid_to_terminate = None
    try:
        for stdout_line in iter(proc.stdout.readline, ''):
            line = stdout_line.decode().strip()
            print(line)
            #look for 'error' in the output
            if "error" in line:
                print("ERROR found in script output")
                pid_to_terminate = proc.pid
                break
    finally:
        proc.stdout.close()
    
    if pid_to_terminate:
        # kill with all spawned processes
        os.killpg(os.getpgid(pid_to_terminate), signal.SIGTERM)
gch
  • 91
  • 5
  • Thank you very much for your answer and suggestions! This is a good solution for monitoring the processes. But in my case, it is difficult to predict the execution time for the program using MATLAB. I think, in my case, it would be better to exit the MATLAB once any errors occurred. – mining Oct 01 '21 at 10:47
  • I edited my answer with new solution: If your Matlab program prints something that can be recognized as an error, you can catch it in python program and then terminate the process. – gch Oct 01 '21 at 12:02
  • Thank you very much for your so detail information. I've updated my post for clarity, please check there. Your answer provide a good solution to terminate all spawned processed in Python. – mining Oct 01 '21 at 12:21