0

Given

  • This code represents some module/service
  • The module receives a group of items each time and start process it
  • The module should's wait for previous groups to complete to start processing the new one (only CPU limitations)

The output of the code below is: 000000000011111111112222222222

Hot to change the code to make next output: 012012012012012012012012012012?

    var executionDataflowBlockOptions =
        new ExecutionDataflowBlockOptions() { 
            MaxDegreeOfParallelism = 3, EnsureOrdered = false, BoundedCapacity = 30 };

    var testData =
        Enumerable
            .Range(0, 3)
            .Select(o => Enumerable.Repeat(o, 10));

    var inputBlock =
        new TransformManyBlock<IEnumerable<int>, int>(
            collectionOfItems => collectionOfItems, 
            executionDataflowBlockOptions);

    var processBlock =
        new ActionBlock<int>(
            number =>
            {
                Thread.Sleep(100);
                Console.Write(number);
            }, 
            executionDataflowBlockOptions );

    inputBlock
        .LinkTo(processBlock);

    foreach (var item in testData)
    {
        await inputBlock.SendAsync(item);
    }

UPDATE (EnsureOrdered)

Finally found this option (thanks JSteward). I used the Microsoft.Tpl.Dataflow (which looks obsolete) instead of System.Threading.Tasks.Dataflow. Not sure why nuget search brings them in a such strange order.

Anyway adding this option changes results. Sometime it outputs 000000000022222222221111111111 (changes order groups), Changing the BoundedCapacity=30 sometime output such results: 210202222222020000000111111111 or 201012222222212111111001000000, which is much better, but still results can be more random

Kamarey
  • 10,832
  • 7
  • 57
  • 70
  • I think you should use `Task.WhenAll`. (I didn't give you the downvote.) – Santhos Oct 17 '17 at 13:49
  • @Kamarey You can set `EnsureOrdered` to `false` and that will allow items to propagate when ever they're ready. But you still won't get `012012` and your desired output seems to contradict your last requirement that groups must wait to be processed. Can you elaborate what exactly you're looking for? – JSteward Oct 17 '17 at 15:01
  • @JSteward: I can't find the EnsureOrdered option (https://msdn.microsoft.com/en-us/library/system.threading.tasks.dataflow.executiondataflowblockoptions(v=vs.110).aspx). I want to start to process the group 3 before group 1 and 2 finish processing. E.g. that the processBlock will take items from all groups. I don't mean the exact order 012012, but the idea that 2 can be processed in parallel with 0 and 1 groups. – Kamarey Oct 17 '17 at 16:01
  • @Kamarey `EnsureOrdered` isn't listed in MSDN as you saw but you can [see it here](https://github.com/dotnet/corefx/blob/master/src/System.Threading.Tasks.Dataflow/src/Base/DataflowBlockOptions.cs). There was also a long discussion on what to name that option but I can't find it at the moment. Just make sure you are using the latest `System.Threading.Tasks.Dataflow` nuget package and the option will be available. If memory serves the option has been available since v4.5.25. Setting that option will allow you to process items in any order. – JSteward Oct 17 '17 at 16:12
  • I remember reading that `EnsureOrdered = false` is an optimization hint ignored by many blocks. If you really need to ensure a random order, that would involve a custom `IPropagatorBlock` with a shuffle algorithm. Trouble is, shuffling a sequence requires its length ahead of time, and thus a big ugly bottleneck in the data flow. – piedar Oct 18 '17 at 07:24

0 Answers0