Tpl implementation issue

Question

Tpl Sample Click here to download

Hi , I am trying to implement tpl in my project . where i have created 5 parallel HTTP call using web client .

What i am trying to achieve. If any of 5 parallel call return me string containing "First" Then kill rest call and continue with call returned "First" .

What i have tried :

I have attached sample code above. where i have used following Predicate function.

async Task<T> WhenAny<T>(IEnumerable<Task<T>> tasks, Func<T, bool> predicate)
    {
        var taskList = tasks.ToList();
        Task<T> completedTask = null;
        do
        {
            completedTask = await Task.WhenAny(taskList);
            taskList.Remove(completedTask);
        } while (!predicate(await completedTask) && taskList.Any());

        return completedTask == null ? default(T) : await completedTask;
    }

And calling this as below:

 public async Task<string> methodname()
    {
        string sUrl = "https://abcd.com/test.php";
        Task<string> task1 = DownLoadData(sUrl);
        Task<string> task2 = DownLoadData(sUrl);
        Task<string> task3 = DownLoadData(sUrl);
        Task<string> task4 = DownLoadData(sUrl);
        Task<string> task5 = DownLoadData(sUrl); 
        var tasks = new[] { task1, task2, task3, task4, task5 };
        await WhenAny(tasks, t => t.Contains("First"));

        return "";


    }

But it don't met criteria. Please suggest where i am missing something. Any help would be appreciated.

Answer 1

I assume that you have a list of urls to check rather than checking the same url repeatedly(?!) but the same method can be used in any event....

I like to use TPL Dataflow for this sort of thing. It lets you chain together multiple async operations (e.g. downloading the string) with multiple sync operations (like checking the content for the substring "First") and provides all manner of 'knobs' to tweak controlling degrees of parallelism, buffer size, etc. We can also pass a cancelation token to cancel further processing. So first let's create a cancellation token source:

var cancellationSource = new CancellationTokenSource();

Now, our first 'block' would be a TransformBlock<string,string>. It's this block's job to take the url string and 'transform' it into a content string by calling your DownloadData function:

var downloadBlock = new TransformBlock<string,string>(
        async s => await DownloadData(s), 
        new ExecutionDataflowBlockOptions
        {
            MaxDegreeOfParallelism = 10, //allow up to 10 simulteneous downloads
            BoundedCapacity = 100,       //allow the block to store up to 100 urls in a buffer
            CancellationToken = cancellationSource.Token 
        });

The second block would be a TransformMany<string,string> block. This type of block allows us to transform a string into a collection of strings. We just give this the job of filtering out any content strings which dont meet our criteria by returning an empty collection:

    var filterBlock = new TransformManyBlock<string,string>(
        s => {
                if (s.Contains("First")) 
                {
                    return new string[]{s};
                } else 
                {
                    return new string[]{};
                }
            }); // (we could also pass a cancellation token here)

The last block is where you would do something with the content string you have found (i.e. the first downloaded content you found that contains "First"). We also Cancel() our cancellation token source to prevent us starting processing of any new urls once we have found one meeting our criteria:

var finalBlock = new ActionBlock<string>(
        s => {
                cancellationSource.Cancel();
                //do whatever you want with the content string here            
            });

All that we need to do now is join up the blocks and feed in the URLs:

downloadBlock.LinkTo(
    filterBlock,
    new DataflowLinkOptions
    {
        PropagateCompletion = true // this flag ensures that if the first block completes ALL urls it signals to the nect block that no more will come
    }
);
filterBlock.LinkTo(
    finalBlock,
    new DataflowLinkOptions
    {
        PropagateCompletion = true,
        MaxMessages = 1         //we are only interested in the first message that arrives in our final block
    });

downloadBlock.Post("http://url.com/blah1");
downloadBlock.Post("http://url.com/blah2");
// etc...

downloadBlock.Complete(); // this signals that we've ran out of Urls to process
finalBlock.Completion.Wait(); //now wait for any messages we have 'in progress' to work through the pipeline