Perl multithreading - thread doesn't start

Question

I need some help, I can't figure out why my thread doesn't want to start. I don't have experience with perl, and was asked to make a script that will process a file row by row. Depending on the row, the process should execute other functions (not in snippet), call the same function on a new file or call the same function on a new file in parallel (thread).

Below, I pasted a snippet of the actual code (removed the non-relevant code).

I'm testing the multithreading part on a function called "test" which should print "ok".

The process executes correctly, "start" is printed, but then it gets stuck and after a brief delay, the process stops executing completely.

A thousand thanks to whoever may help me!

use strict;
use warnings;
use IO::Prompter;
use Getopt::Long;
use Log::Message::Simple;
use File::Basename;
use File::Spec;
use IO::Socket::INET;
use UUID::Tiny ':std';
use threads;
use threads::shared;

# *bunch of code deleted*

process_file( $cmdline{csvfile}, 1 );

sub test {
    print "ok\n";
}

sub process_file {

    # get parameters
    my ( $input_file, $flowid ) = @_;

    # init variables

    # open input file
    open( my $fh, '<:encoding(UTF-8)', $input_folder . $input_file )
        or die "Could not open file '$input_file' $!";

    # process file
    while ( my $row = <$fh> ) {
        chomp $row;
        @request   = split ";", $row;
        $flow_type = $request[0];
        $flow      = $request[1];

        # *bunch of code deleted*

        $filename    = "$flow.csv";
        $keep_flowid = $request[2];                       # keep flowid?
        $tmp_flowid  = $keep_flowid ? $flowid : undef;    # set flowid
        $thread      = $request[3];
        if ( $thread == 1 ) {
            ### Create new thread
            print "start\n";
            my $process_thread = threads->create("test");
            $process_thread->join();
        }
        elsif ( $thread == 0 ) {

            # wait on process to complete
            process_file( $filename, $tmp_flowid );
        }

        # *bunch of code deleted*

    }

    # close file
    close $fh or die "Couldn't close inputfile: $input_file";
}

Unfortunately the use of threads is [discouraged](http://perldoc.perl.org/threads.html#WARNING) since Perl 5.20. I like threading in Perl, especially when used together with `Threads::Queue`. However: Your `test` function is so small (and quick) that it likely already has returned when you call `join()` on it. Also you cannot benefit from threads when you start them and then wait for them (with `join`) to finish. — PerlDuck, May 10 '16 at 19:20
From the code I assume you don't `use strict` nor `use warnings`. Please do so. Did you include `use threads;`? — PerlDuck, May 10 '16 at 19:32
Thanks @PerlDog Is there something else encouraged to handle parallel execution in perl? I wrote another script using multithreading in perl 5.22 last week, but it seems repeating is more difficult :) Unfortunately, the structure is completely different, so I can't copy paste.. The `join` is indeed misplaced (for debugging purposes). Do you have an idea why the thread won't start though? — Jonas De Ridder, May 10 '16 at 19:45
Yes, I do use those :), there's quite some code above. Also, I don't get any errors/warnings on execution. — Jonas De Ridder, May 10 '16 at 19:47
@Perl Dog, Perl threads work, they are maintained, and they're not about to be removed. // They're supposedly only discouraged if you don't know how to deal with the threads being heavy. In other words, if you don't mind the thread taking some time to start up, or if you're using a worker model, there's no problems using threads. // Basically, a group got tired of seeing newcomers waste CPU time (in their minds) by using threads or by not reusing them. — ikegami, May 10 '16 at 20:24
@Jonas De Ridder, If it's expensive to start a thread, then having each thread perform more than one work unit will be faster than creating a new thread for each work unit. This is the concept behind worker pools — ikegami, May 11 '16 at 00:44
http://stackoverflow.com/questions/26296206/perl-daemonize-with-child-daemons — Sobrique, May 11 '16 at 07:53
It may not start, because you need an id of 1 for it to do so. — Sobrique, May 11 '16 at 07:55
@ikegami Great to hear threads are still maintained. :-) I've several scripts that start N workers which get their "jobs" fed through a `Thread::Queue`. It works so smoothly and is so simple that I'd prefer not to replace it with Parallel::Forkmanager. Thanks! — PerlDuck, May 11 '16 at 09:45

score 1 · Answer 1 · edited May 23 '17 at 11:59

It's hard to say exactly why you're having this problem - the major possiblity seems to be:

    $thread = $request[3];
    if ($thread == 1){

This is input from your filehandle, so a real possiblity is that "$request[3]" isn't actually 1.

I am a bit suspicious though - your code as use strict; use warnings at the top, but you're not declaring e.g. $thread, $flow etc. with my. That either means you're not using strict, or you're reusing variables - which is a good way to end up with annoying glitches (like this one).

But as it stands - we can't tell you for sure, because we cannot reproduce the problem to test it. In order to do this, we would need some sample input and a MCVE

To expand on the point about threads made in the comments - you may see warnings that they are "Discouraged". The major reason for this, is because perl threads are not like threads in other languages. They aren't lightweight, where in other languages they are. They're perfectly viable solutions to particular classes of problems - specifically, the ones where you need parallelism with more IPC than a fork based concurrency model would give you.

Hi @Sobrique , I declare those variables in the code not showed, I'm sure this isn't the problem as "start" is printed to console (also, I use `strict` & `warnings` .and get no error message). Thanks for pointing out the `forks`, I'll take a look and if no luck, I'll create an MCVE & sample input so you can test the thread-code. — Jonas De Ridder, May 11 '16 at 10:08
One obvious alternative - do you have another `test` within your namespace? What happens if you `create(\&test)` instead? Declaring the variables elsewhere isn't a good practice as a rule - use them within the minimum scope you can, and you'll avoid 'bleed' (e.g. var reuse). — Sobrique, May 11 '16 at 10:40

score 0 · Answer 2 · answered May 12 '16 at 15:49

0

I suspect you are experiencing this bug, fixed in Perl 5.24.

If so, you could work around it by performing your own decoding rather than using an encoding layer.

answered May 12 '16 at 15:49

ikegami

367,544
15
269
518

Perl multithreading - thread doesn't start

2 Answers2