Behaviour of feed operator pipeline

Question

I'm experimenting with the feed operator using the following code:

my $limit=10000000;
my $list=(0,1...4);
sub task($_,$name) {
    say "Start $name on $*THREAD";
    loop ( my $i=0; $i < $limit; $i++ ){};
    say "End $name";
    Nil;
}

sub stage( &code, *@elements --> Seq) {
    map(&code,  @elements);
}

sub feeds() {
    my @results;
    $list
    ==> map ({ task($_, "stage 1"); say( now - ENTER now );$_+1})
    ==> map ({ task($_, "stage 2"); say( now - ENTER now );$_+1})
    ==> stage ({ task($_, "stage 3"); say( now - ENTER now );$_+1}) 
    ==> @results;

    say @results;
}

feeds();

the task sub is just a loop to burn cpu cycles and displays the thread used
the stage sub is a wrapper around the map sub
each step of the feed pipeline maps to calling the cpu intensive task and timing it. The result of the map is the input +1

The output when run is:

Start stage 1 on Thread<1>(Initial thread)
End stage 1
0.7286811
Start stage 2 on Thread<1>(Initial thread)
End stage 2
0.59053989
Start stage 1 on Thread<1>(Initial thread)
End stage 1
0.5955893
Start stage 2 on Thread<1>(Initial thread)
End stage 2
0.59050998
Start stage 1 on Thread<1>(Initial thread)
End stage 1
0.59472201
Start stage 2 on Thread<1>(Initial thread)
End stage 2
0.5968531
Start stage 1 on Thread<1>(Initial thread)
End stage 1
0.5917188
Start stage 2 on Thread<1>(Initial thread)
End stage 2
0.587358
Start stage 1 on Thread<1>(Initial thread)
End stage 1
0.58689858
Start stage 2 on Thread<1>(Initial thread)
End stage 2
0.59177099
Start stage 3 on Thread<1>(Initial thread)
End stage 3
3.8549498
Start stage 3 on Thread<1>(Initial thread)
End stage 3
3.8560015
Start stage 3 on Thread<1>(Initial thread)
End stage 3
3.77634317
Start stage 3 on Thread<1>(Initial thread)
End stage 3
3.6754558
Start stage 3 on Thread<1>(Initial thread)
End stage 3
3.672909
[3 4 5 6 7]

The result in @result is correct (The input from $list in incremented 3 three times)

The output of the first two stages pulls/alternates but the third stage doesn't execute until the completion of all the input into stage 2

Is there an issue with my wrapper to the map sub causing this behaviour?

Also the time it takes to evaluate in the wrapper is significantly longer than a direct call to map.

Any help is appreciated.

I would help if you golfed it down. Some things can be eliminated, for instance, the reference to a thread, since it's all running in the same thread. Also, what you are returning is a Seq; probably they are not itemized until they are really needed to, which would be when you arrive to third stage. I would say that's normal Seq behavior. — jjmerelo, Mar 15 '19 at 13:59
@ugexe you nailed it. Changing `*@elements` with `*$elements` lets the third stage run instep with the others instead of waiting until they complete. As a mentioned below, I was attempting to match `map`s signature, which was using a `*@` (at least the doc I found). If you would like to post an answer I will except it. thanks — drclaw, Mar 15 '19 at 21:27
@jjmerelo My golf game needs some work! I was rushing out the door when I originally posted. I'll try format my questions better in the future. Thanks for your feedback — drclaw, Mar 15 '19 at 21:31

ugexe · Answer 1 · 2023-03-16T15:53:34.797

6

The slurpy parameter is waiting to consume the entire sequence being passed to it. Consider the difference between:

raku -e 'sub foo($) { }; my $items := gather for 1..Inf { say $_ }; foo($items);'

raku -e 'sub foo(*@) { }; my $items := gather for 1..Inf { say $_ }; foo($items);'

The first example will finish, the second one will never finish. Thus you would want to change your stage function to:

sub stage( &code, $elements --> Seq) {
    map(&code,  $elements);
}

edited Mar 16 '23 at 15:53

answered Mar 15 '19 at 19:32

ugexe

5,297
1
28
49

The reason I used a slurpy was an attempt to match the signature of the `map` sub https://docs.perl6.org/routine/map#class_List. I removed the slurpy and the execution time is now consistent! thanks. However the third stage still does not execute until the first two stages are complete. – drclaw Mar 15 '19 at 21:09
1

@drclaw Itemization, probably, as I commented above. – jjmerelo Mar 15 '19 at 21:47

Behaviour of feed operator pipeline

1 Answers1