Parallel Programming Interlocked C#

Question

Im confused with the example at

http://msdn.microsoft.com/en-us/library/dd997393.aspx

Parallel.ForEach<int, long>(nums, // source collection
                                    () => 0, // method to initialize the local variable
                                    (j, loop, subtotal) => 
                                    {
                                        subtotal += nums[j]; 
                                        return subtotal; 
                                    },

                                    (finalResult) => Interlocked.Add(ref total,finalResult)                                        );

I dont know why the last delegate (finalResult) => Interlocked.Add(ref total,finalResult) requires an Interlock, whereas the previous expression

(j, loop, subtotal) => 
                                    {
                                        subtotal += nums[j]; 
                                        return subtotal; 
                                    },

does not?

Thanks

Answer 1

The Parallel.For() and Parallel.ForEach() methods make use of a Partitioner. It would be very inefficient to execute a loop over 10,000 elements on 10,000 individual Tasks. The partitioner splits the data in segments and ideally the ForEach() will execute in 4 Tasks (threads) of 2,500 elements on a 4-core CPU. This sometimes requires some heuristics and you can write your own custom-partitioner.

When using the 'normal' (simple) overloads of ForEach() this is fully transparent. But your example uses one of the <TLocal> overloads that surfaces the partitioning.

The subtotal += nums[j]; statement is iterated inside 1 partition and is therefore thread-safe.

And (finalResult) => Interlocked.Add(ref total,finalResult) is where the partitions are merged, this part is of course not thread-safe.

Answer 2

The variable subtotal is "local data" that only one thread has access to.

On the other hand, total is a variable that all the threads can modify, and potentially several threads may try to update the total at the same time.