ConcurrentBag(); with duplicates in Parallel.Foreach

Question

I have to read some record from DB, do some work with the info and back the new info as an Update to the Db, in one method I receive the datatable and using a Parallel.ForEach I am trying to speed up the process, I am using a switch to define what is the operation to do with the current row in the loop and sending the row as a parameter to the static method, the issue I am facing is that some duplicated records are added to the resultCollection = new ConcurrentBag<DataRow>();

Here is my code:

private void DoSomething(  DataTable dataTable  )
{
    var storage2 = new ConcurrentBag<DataRow>();

    Parallel.ForEach(dataTable.AsEnumerable(), new ParallelOptions { MaxDegreeOfParallelism = 4 }, dRow =>
    {
     int rand_num = GetRandomNumber(1, 5); //select the method base in random
     try
      {          
        var res = rand_num switch
        {
         1 => NewMethod1(dRow ),
         2 => NewMethodAsync2 (dRow ),
         3 => NewMethodAsync3(dRow ),
        _ => NewMethodAsync4(dRow )
       };

      storage2.Add(res)
     }
     if (!storage2.IsEmpty)
       {
         var paramTable = new DataTable();
         paramTable = dataTable.Clone();
         paramTable = storage2.CopyToDataTable(); // Copy concurrent bag to table
         storage2 = new ConcurrentBag<DataRow>(); // Delete all content in concurrent bag

         UpdateDatabaseTable (paramTable); // **Error while updating because has some duplicated rows**
       }
}

This is the NewMethod1,2,3,4 code, all methods do the same

private static DataRow NewMethod1(DataRow row )
{
     DataRow newRow = paramTable2.NewRow();
      try
        { 
          // change the value
           newRow[0] = row[0];
           newRow[1] = row[1];
           newRow[1] = 'The new value';
           newRow[1] = 'Another new value';
        }
        return newRow;
}

The duplicates are intermittent issue, this issues is not happening with MaxDegreeOfParallelism = 1 only when is greater or -1.

Could you please share with us valid C# code? According to my knowledge you can't use `try` alone without `catch` or `finally` block(s). — Peter Csala, Feb 14 '22 at 10:09
Parallelizing slow queries won't make them faster, it will only *increase* delays due to blocking. What is the actual UPDATE query that you want to accelerate? You won't make it go faster if you try to execute it multiple times in parallel - the database is using the same disks, same RAM, same network card and won't go faster if you try to execute the same thing in parallel. *Better indexes* will improve performance. Batching multiple statements will improve performance by eliminating the network overhead. Using a more efficient query instead of individual statements will be even faster — Panagiotis Kanavos, Feb 14 '22 at 10:12
Somewhat related: [Thread safety for DataTable](https://stackoverflow.com/questions/21310757/thread-safety-for-datatable) — Theodor Zoulias, Feb 14 '22 at 11:28

ConcurrentBag(); with duplicates in Parallel.Foreach

0 Answers0