Combining PLINQ with Async method

Question

I'm trying to combine my PLINQ statement like this:

Enumerable.Range(0, _sortedList.Count()).AsParallel().WithDegreeOfParallelism(10)
          .Select(i =>  GetTransactionDetails(_sortedList[i].TransactionID))
          .ToList();

With an async method like this:

 private async void GetTransactionDetails(string _trID)
 {
      await Task.Run(() =>
      {
      });
 }

So that I can simply add an await operator in here:

 Enumerable.Range(0, _sortedList.Count()).AsParallel().WithDegreeOfParallelism(10)
           .Select(i => await GetTransactionDetails(_sortedList[i].TransactionID))
           .ToList();

How can I achieve this ?

P.S. This way I could make 5-10 HTTP requests simultaneously while ensuring that the end user doesn't feels any "screen" freezing while doing so...

Why are you trying to parallelize the *starting* of an asynchronous operation, given that starting it is going to take basically zero time at all. PLINQ is for long running CPU bound operations; that's not what you have. Additionally it looks like `GetTransactionDetails` is using the async over sync anti-pattern. It shouldn't offload the work to another thread, rather it should just be a synchronous method. If the caller wants to call it with `Task.Run`, they can, if they want to do something else, like use PLINQ, then they could do that. — Servy, Dec 13 '16 at 16:46
@Servy could you show my how could I implement that what you just said on a more practical example ? — User987, Dec 13 '16 at 16:48
Remove `Task.Run` from `GetTransactionDetails`, and just have it do the work synchronously, thereby allowing PLINQ to parallelize it. — Servy, Dec 13 '16 at 16:49
Oh like that... I can't really do that since I make like 800 requests over the span of 10-15 minutes to PayPal, while doing so, the application completely freezes .. :/ Is there any way of avoiding the application freezing while doing the PLINQ statement ? — User987, Dec 13 '16 at 16:50
If the work being done is actually network requests, then you shouldn't be using `Task.Run` at all, the requests should inherently be `Task` returning, and there is no reason at all to be using PLINQ, since you don't have CPU operations you want to perform synchronously, you should simply `await` the network requests that you have. — Servy, Dec 13 '16 at 16:54
@Servy So basically I should just turn my method into async Task and then do an await operator? If I do that, how do I fetch the value returned by the method itself. Even bigger question, what would the method now return if I made it async Task ? — User987, Dec 13 '16 at 16:57
Sounds like you need to go read an intro tutorial on asynchronous programming so that you can get the basics down. It's beyond the scope of what can be done in an SO post to go over that kind of information. — Servy, Dec 13 '16 at 16:58
@Servy I've updated my initial question with the updated function , is this it ? — User987, Dec 13 '16 at 17:01

score 19 · Accepted Answer · answered Dec 18 '16 at 00:30

19

There's a couple approaches you can take.

First, the "more correct" approach (which is also more work). Make GetTransactionDetails into a proper async method (i.e., does not use Task.Run):

private async Task GetTransactionDetailsAsync(string _trID);

Then you can call that method concurrently:

var tasks = _sortedList.Select(x => GetTransactionDetailsAsync(x.TransactionID));
await Task.WhenAll(tasks);

If you need to limit concurrency, use a SemaphoreSlim.

The second approach is more wasteful (in terms of thread usage), but is probably easier given what parts of your code we've seen. The second approach is to leave the I/O all synchronous and just do it in a regular PLINQ fashion:

private void GetTransactionDetails(string _trID);

_sortedList.AsParallel().WithDegreeOfParallelism(10).Select(x => GetTransactionDetails(x.TransactionID)).ToList();

To avoid blocking the UI thread, you can wrap this in a single Task.Run:

await Task.Run(() => _sortedList.AsParallel().WithDegreeOfParallelism(10).Select(x => GetTransactionDetails(x.TransactionID)).ToList());

answered Dec 18 '16 at 00:30

Stephen Cleary

437,863
77
675
810

Clearly Amazing reply , thanks ! The await Task.Run() Is exactly what I needed in this case... Although I had to use Parallel.For with combination of Parallel.For loop instead of the one you've shown in the example because .Select method doesn't work with void methods, at least thats what the compiler shown me ? – User987 Dec 18 '16 at 18:21
I have CPU-bound async methods which do a tiny bit of IO but need to be parallelized for the CPU-bound stuff. The interfaces it uses don't have a synchronous API. What I did was `Task.WhenAll(items.Select(item => Task.Run(async () => { /* method which awaits */ })))`. The Task.Run is to handle the CPU-bound work which happens before the first await. Queuing all at once to the thread pool seems to be the fastest so far. – jnm2 Jul 10 '17 at 21:11
People running into @jnm2 's use-case (CPU-bound work) should look into PLINQ – miniBill Nov 28 '17 at 09:02
3

@miniBill The problem with PLINQ is that it's not great for CPU-bound with intermittent IO bounds which is my use case here. It forces you to block on async IO. – jnm2 Nov 28 '17 at 12:18

Combining PLINQ with Async method

1 Answers1

Linked