1

I have a python program that starts a number of processes. And these processes in turn spawn their own subprocesses (nested processes). I like to keep a list of all processes started by either the main or any of the subprocesses, but could not figure out how to get the handle of the nested processes back to the main. This is what I tried but the compiler says process handles are not pickable.

import time
import sys
import os
import multiprocessing
import subprocess

def thread2(ns):
    loopcnt = 0

    extCmd = os.path.join("/bin/sleep 20")
    nestedprocess = subprocess.Popen(extCmd.split(' '))

    if nestedprocess.poll() is None:  # check if process has started.
        print("{} cmd started.".format(extCmd))
        ns.nestedprocess = nestedprocess  # <=== TypeError: can't pickle _thread.lock objects
    else:
        print("Error: {} failed to start.".format(extCmd))

    while loopcnt < 5:
        loopcnt += 1
        print("Thread running: {}".format(loopcnt))
        ns.data += loopcnt
        time.sleep(0.1)

    print("Thread2 done.")
    return


if __name__ == "__main__":
    namespace = multiprocessing.Manager().Namespace()
    namespace.data = 10  # int is pickable so there are no problems.
    namespace.nestedprocess = None

    process2 = multiprocessing.Process(target=thread2, args=(namespace, ))
    process2.start()
    process2.join()

    print("Data: {}.".format(namespace.data))
    print("process: {}.".format(namespace.nestedprocess))
    sys.exit(0)
Chu Bun
  • 513
  • 1
  • 5
  • 17
  • You can't. Also, they're not threads, which is the biggest reason *why* you can't. – user2357112 Jan 04 '20 at 01:05
  • So are there any way for the main process to reference nonpickable vars from a subprocess? – Chu Bun Jan 04 '20 at 01:10
  • By the way, does the nestedprocess variable has any meaning in the context of the main thread? Event if I could somehow pass it back to main, can main uses it to terminate the running external program? – Chu Bun Jan 07 '20 at 17:06

0 Answers0