How to spawn multiple python scripts from a python program?

Question

I want to spawn (fork?) multiple Python scripts from my program (written in Python as well).

My problem is that I want to dedicate one terminal to each script, because I'll gather their output using pexpect.

I've tried using pexpect, os.execlp, and os.forkpty but neither of them do as I expect.

I want to spawn the child processes and forget about them (they will process some data, write the output to the terminal which I could read with pexpect and then exit).

Is there any library/best practice/etc. to accomplish this job?

p.s. Before you ask why I would write to STDOUT and read from it, I shall say that I don't write to STDOUT, I read the output of tshark.

score 5 · Answer 1 · edited Jun 20 '20 at 09:12

5

See the subprocess module

The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several other, older modules and functions, such as:

os.system

os.spawn*

os.popen*

popen2.*

commands.*

edited Jun 20 '20 at 09:12

Community

1
1

answered Jun 07 '09 at 13:12

gimel

83,368
10
76
104

score 1 · Answer 2 · answered Jan 11 '20 at 13:34

1

From Python 3.5 onwards you can do:

    import subprocess

    result = subprocess.run(['python', 'my_script.py', '--arg1', val1])
    if result.returncode != 0:
        print('script returned error')

This also automatically redirects stdout and stderr.

answered Jan 11 '20 at 13:34

Shital Shah

63,284
17
238
185

score 0 · Answer 3 · answered Jul 12 '14 at 17:14

You want to dedicate one terminal or one python shell?

You already have some useful answers for Popen and Subprocess, you could also use pexpect if you're already planning on using it anyways.

#for multiple python shells
import pexpect

#make your commands however you want them, this is just one method
mycommand1 = "print 'hello first python shell'"
mycommand2 = "print 'this is my second shell'"

#add a "for" statement if you want
child1 = pexpect.spawn('python')
child1.sendline(mycommand1)

child2 = pexpect.spawn('python')
child2.sendline(mycommand2)

Make as many children/shells as you want and then use the child.before() or child.after() to get your responses.

Of course you would want to add definitions or classes to be sent instead of "mycommand1", but this is just a simple example.

If you wanted to make a bunch of terminals in linux, you just need to replace the 'python' in the pextpext.spawn line

Note: I haven't tested the above code. I'm just replying from past experience with pexpect.

score 0 · Answer 4 · answered Jun 09 '09 at 14:15

I don't understand why you need expect for this. tshark should send its output to stdout, and only for some strange reason would it send it to stderr.

Therefore, what you want should be:

import subprocess

fp= subprocess.Popen( ("/usr/bin/tshark", "option1", "option2"), stdout=subprocess.PIPE).stdout
# now, whenever you are ready, read stuff from fp

How to spawn multiple python scripts from a python program?

4 Answers4