5

I'm trying to understand how parallelism might work using PLINQ, given delayed execution. Here is a simple example.

string[] words = { "believe", "receipt", "relief", "field" };
bool result = words.AsParallel().Any(w => w.Contains("ei"));

With LINQ, I would expect the execution to reach the "receipt" value and return true, without executing the query for rest of the values.

If we do this in parallel, the evaluation of "relief" may have began before the result of "receipt" has returned. But once the query knows that "receipt" will cause a true result, will the other threads yield immediately?

In my case, this is important because the "any" test may be very expensive, and I would want to free up the processors for execution of other tasks.

tbischel
  • 6,337
  • 11
  • 51
  • 73

1 Answers1

5

Unfortunately, the other threads will not "yield" immediately.

As soon as Any() finds a valid element, the PLINQ scheduler will stop scheduling new threads to check for new elements. Any existing partitioners will also receive a cancellation request, which will prevent those partitions from calling Any() on another item.

However, any threads that are currently executing the lambda expression within your Any() method will still be executing, as there's no way for them to know that another thread has succeeded. It will prevent new threads from calling into Any(), but not cancel all of the ones in a "very expensive" delegate.

On a side note:

PLINQ, unlike LINQ to Objects, doesn't really use deferred execution. When you call AsParallel() on an IEnumerable<T>, the ParallelQuery<T> that is generated will actually start processing your routine in parallel. Deferred execution would dramatically reduce the effectiveness of PLINQ, since it would be impossible to schedule, in parallel, without creating the work partitioners and scheduling in advance.

Edit:

After thinking about this - if your lambda is VERY expensive, you might want to consider using a CancellationToken. I blogged, in detail, about how cancellation in PLINQ works. Typically, you'd just use a token and call ThrowIfCancellationRequested() - however, you can also use a CancellationToken and check IsCancellationRequested, which would let you make your lambda "exit early", providing you a way to stop the background processing sooner...

Reed Copsey
  • 554,122
  • 78
  • 1,158
  • 1,373
  • 2
    http://msdn.microsoft.com/en-us/library/dd997425(VS.100).aspx says deferred execution principles are still in play with PLINQ... can you clarify a bit what you mean in your side point? – tbischel Mar 08 '10 at 19:17
  • 2
    @tbischel: They do, and they don't... That comment is slightly misleading. The difference is that, in LINQ, each element is only executed as requested (deferred). In PLINQ, as soon as you make the FIRST request, a `Partitioner` is setup, which begins scheduling out your work to multiple threads. The final accumulated results are not pushed back until requested, but processing happens prior to the request for an element. (If you request element 1 of the results, elements 1,2,3&4 may all get scheduled, and start "working" immediately...) – Reed Copsey Mar 08 '10 at 19:33