Parallelizing the Processing of an Array

Question

sub parallelizing{
  my counter = 0; 
  my $MAX_PROCESS = 10;
  my $workerQueue = Parallel::ForkManager->new($MAX_PROCESS);

  $workerQueue->start and next;
    print "$process #" . $counter . " started\n";
    $counter = $counter +1
  $workerQueue->finish;
}

I am using Parallel::ForkManager to create child processes that share the variable $counter, but it turns out that it's not shared. Is there any way to let child processes to share a variable?

You would need to have them talk to the parent process and have the parent assign the element. It may be easier to pre-slice the array and hand chunks to each worker. — simbabque, Sep 17 '15 at 14:27
Use [forks::shared](http://p3rl.org/forks::shared) or [threads::shared](http://p3rl.org/threads::shared) if you need to share data. — choroba, Sep 17 '15 at 14:32
That code won't compile for a couple of reasons. Please post the *real* code that you have written and tested — Borodin, Sep 17 '15 at 15:31
I intend to parallelize the processing of a big array, and wanted to emulate the problem of changing slice index by the code snippet posted. Now I realize it's easier and enough to pass the changed index from the parent to the child instead of attempting to change it in the child process. — CCNA, Sep 17 '15 at 19:17

score 0 · Answer 1 · edited May 23 '17 at 10:27

Short answer - no, not that way. fork creates a separate process instance, with it's own memory state. It implements a copy-on-write mechanism (generally) so it's quite efficient. But processes can't share memory trivially.

However you can do something like threads::shared to create a threaded; shared variable accessible by multiple things. (Although I'd normally suggest using Thread::Queue instead).

Something like this: Perl daemonize with child daemons Might give some useful examples.

Parallelizing the Processing of an Array

1 Answers1