0

I want to use use multiprocessing to do the following:

class myClass:

    def proc(self):
        #processing random numbers
        return a

    def gen_data(self):
        with Pool(cpu_count()) as q:
            data = q.map(self.proc, [_ for i in range(cpu_count())])#What is the correct approach?
        return data
martineau
  • 119,623
  • 25
  • 170
  • 301
Frankie
  • 744
  • 1
  • 9
  • 14

2 Answers2

0

Try this:

def proc(self, i):
    #processing random numbers
    return a

def gen_data(self):
    with Pool(cpu_count()) as q:
        data = q.map(self.proc, [i for i in range(cpu_count())])#What is the correct approach?
    return data
Chan
  • 3,605
  • 9
  • 29
  • 60
0

Since you don't have to pass an argument to the processes, there's no reason to map, just call apply_async() as many times as needed.

Here's what I'm saying:

from multiprocessing import cpu_count
from multiprocessing.pool import Pool
from random import randint

class MyClass:

    def proc(self):
        #processing random numbers
        return randint(1, 10)

    def gen_data(self, num_procs):
        with Pool() as pool:  # The default pool size will be the number of cpus.
            results = [pool.apply_async(self.proc) for _ in range(num_procs)]
            pool.close()
            pool.join()  # Wait until all worker processes exit.

        return [result.get() for result in results]  # Gather results.

if __name__ == '__main__':
    obj = MyClass()
    print(obj.gen_data(8))
martineau
  • 119,623
  • 25
  • 170
  • 301