1

I tested a code using python 3.6 and python 3.7 by 2 cases.

1 case) using Member variables of class

import multiprocessing as mp
import sys
class Foo:
    def do_multiprocessing(self):
        ctx = mp.get_context('spawn')
        self.process_1 = ctx.Process(target=self.do_stuff1)
        self.process_2 = ctx.Process(target=self.do_stuff2)

        self.process_1.start()
        self.process_2.start()

    def do_stuff1(self):
        print("Doing 1")

    def do_stuff2(self):
        print("Doing 2")

if __name__ == "__main__":
    print("PYTHON VERSION : ", sys.version)
    foo = Foo()
    foo.do_multiprocessing()

Result : When i executed it using python 3.6, working well. However, when i executed it using python 3.7, a error occurred.(TypeError: Can't pickle weakref object)

2 case) using local variables

import multiprocessing as mp
import sys
class Foo:
    def do_multiprocessing(self):
        ctx = mp.get_context('spawn')
        process_1 = ctx.Process(target=self.do_stuff1)
        process_2 = ctx.Process(target=self.do_stuff2)

        process_1.start()
        process_2.start()

    def do_stuff1(self):
        print("Doing 1")

    def do_stuff2(self):
        print("Doing 2")

if __name__ == "__main__":
    print("PYTHON VERSION : ", sys.version)
    foo = Foo()
    foo.do_multiprocessing()

Result : When i executed it using python 3.6 and python 3.7, working well.

Why can't execute multiprocess using member variables of class? And why can execute multiprocess using local variables?

I can't understand why a code execute.

p.s.) if using mp.get_context('fork'), the code works well using python 3.6 and python 3.7 on case 1.

DongHwan Shin
  • 93
  • 8

1 Answers1

0

By passing self.do_stuff2 to process_2, you’re trying to give that process access to the entirety of self (as an implicit argument), which includes self.process_1. You can see how giving one child process meaningful access to another might be complicated (unless you have the OS replicate the whole (parent) process with 'fork').

Davis Herring
  • 36,443
  • 4
  • 48
  • 76
  • Thank you for your reply. Could you explain more easily? I have a hard time understanding it. – DongHwan Shin Feb 07 '22 at 03:02
  • An object `x` is reachable from a bound method created from it (`b=x.f`). Of course it is; otherwise how would calling it (`b()`) provide `self` to `f`? That means, though, that `x.process_1` is also reachable, so trying to launch `x.do_stuff2` as a child process involves trying to send that child process the value `x.process_1`, which it can't do. – Davis Herring Feb 07 '22 at 04:09