Perl - multithreading on foreach loop:

Question

I have a big code that somewhere in the middle of it, there's a foreach loop filling up some dictionaries. Each iteration is independant and fills those dictionaries with disjoint keys.

I'm trying to turn the "foreach" loop in the middle to multithreaded in order to decrease time.

In the following example, $a1, $b1 are the pointers to dictionaries.

I tried "thread::shared" this way:

my $a1 = {};    
my $b1 = {};
my $c1 = {};
my $d1 = {};

# a lot of code using $a1 and $b1

share($a1);
share($b1);
share($c1);
share($d1);
my @threads;
foreach my $Inst ( sort keys %{ $a1->{ports} }) {
            push( @threads, threads->create('some_func', $Inst, $a1, $b1, $c1, $d1, $e ...)); 
}
for my $thr (@threads) {
          thr->join();
}

# all the other code

But I get an error of:

Invalid value for shared scalar at ...

Any ideas how to get the data-structures filled, but not that it would interfere with the code before and after the for-each loop?

Answer 1

It is not possible to make a hash shared after it has been created/declared without losing the data in the hash. Instead you could try use shared_clone() like this:

use feature qw(say);
use strict;
use warnings;
use threads ;
use threads::shared ;
use Data::Dumper qw(Dumper);

my %h1 = (a => 1, b => 2);
my %h2 = (c => 3, d => 4);

my $a1 = \%h1;
my $b1 = \%h2;

my $a1c = shared_clone($a1);
my $b1c = shared_clone($b1);
my $lockvar:shared;

my $nthreads = 3;
for ( 1..$nthreads ) {
    threads->create('job_to_parallelize', $a1c, $b1c, \$lockvar ) ;
}
$_->join() for threads->list();

sub job_to_parallelize {
    my ($a1, $b1, $lockvar) = @_;
    {
        lock $lockvar;
        $a1->{a}++;
        $b1->{d}++;
    }
}

print Dumper({a1c => $a1c});
print Dumper({b1c => $b1c});

Output:

$VAR1 = {
          'a1c' => {
                     'a' => 4,
                     'b' => 2
                   }
        };
$VAR1 = {
          'b1c' => {
                     'd' => 7,
                     'c' => 3
                   }
        };