Multithreaded foreach

Question

I have list of ReferenceIDs (string) in a dataset. These ReferenceIDs can have values like this ("CQ1258891","CQ1258892","CQ1258893"....""CQ1258993"). I have a logic in my code to send a mail for each ReferenceIDs.

As of now I am looping through the every ReferenceID synchronously. Due to which, it take more time to send every mail. I have been using .NET 3.0, so I dont have option to use TPL in .NET 4.0.

I have been looking for a multithreaded machanism to send the mails for every ReferenceID asynchronously. As of now, I have tried the following code but it's not working as expected.

foreach (DataRow row in qrefSet.Tables[0].Rows)
{
    string refId = Convert.ToString(row["ReferenceID"]);
    if (!string.IsNullOrEmpty(refId))
    {
        Thread thread = new Thread(() => apeDBAdapter.SendEmail(personId, refId, parentReferenceID, customerName, queueId));                                    
        thread.Start();
    }
}

Please share the effective machanism to achieve multithreaded foreach loop for my implementation.

Thanks, Sriram

Why not using Threads from thread pool ? read at http://msdn.microsoft.com/en-us/library/system.threading.threadpool.queueuserworkitem.aspx - Let them do the work of thread management — ilansch, Oct 16 '13 at 06:35
You should at least move up to .NET 3.5. "Support for the .NET Framework 3.x versions prior to 3.5 SP1 will remain supported until July 12th, 2011. After this date, support will end; we strongly encourage customers to migrate to .NET Framework 3.5 SP1 before July 12th, 2011." — Damien_The_Unbeliever, Oct 16 '13 at 06:36
In the case of .NET 3.5, how will it be achieved? I am curious to know the right way to do this. — Sriram B, Oct 16 '13 at 06:44
Creating threads is expensive. use ThreadPool.QueueUserWorkItem to queue your jobs — Carlos Landeras, Oct 16 '13 at 06:46
ilansch & Carlos, I tried the ThreadPool.QueueUserWorkItem. I have multiple parameters to pass but this method takes only two parameters. Unable to pass the remainging parameters! Any idea how can I do it? These are my parameters (personId, refId, parentReferenceId, customerName, queueId) — Sriram B, Oct 16 '13 at 06:46
It is possible, ive done it many times, no time to paste sample, but google it there are plenty samples out there ! E.g = http://stackoverflow.com/questions/5564737/threadpool-queueuserworkitem-with-function-argument — ilansch, Oct 16 '13 at 06:51

score 2 · Answer 1 · answered Oct 16 '13 at 06:46

2

Try this

foreach (DataRow row in qrefSet.Tables[0].Rows)
{
    string refId = Convert.ToString(row["ReferenceID"]);
    if (!string.IsNullOrEmpty(refId))
    {
        Thread thread = new Thread(new ParameterizedThreadStart(SendMail));
        thread.Start(refId)           
    }
}
..........
void SendMail(object refId)
{
    string strRefId = (string)refId;
    apeDBAdapter.SendEmail(personId, refId, parentReferenceID, customerName, queueId));  
}

hope it helps...

answered Oct 16 '13 at 06:46

Virus

2,445
1
14
17

But he asked to parallel the foreach...not the actual sending. – Yair Nevet Oct 16 '13 at 06:56
the foreach can be made parallel if he uses TPL, but unfortunately thats not possible for him... so this is the option i would suggest for him. – Virus Oct 16 '13 at 07:05
So if, for example, there are 1000 references, 1000 threads will be started? – Yair Nevet Oct 16 '13 at 07:50
It depends, if there are going to be a bigger bunch of rows... then we need to find some other way... The volume of rows is not mentioned with the question. – Virus Oct 16 '13 at 08:36

Yair Nevet · Answer 2 · 2013-10-16T08:16:44.070

You can create a method that takes a range of items from the rows collection and run it multi-threaded according to the number of devision.

For example:

SendEmailsToReferencesByRange(int start, int end, DataRowCollection rows)
{
  foreach(var item in rows.Range(start, end-start))
  {
     //Do your sending logic here
  }
}

Usage:

//if for example you have 200 items than you can run it in 2 threads
Thread thread = new Thread(()=>SendEmailsToReferencesByRange(0,100, table[0].Rows)).Start();
Thread thread = new Thread(()=>SendEmailsToReferencesByRange(100,200, table[0].Rows)).Start();

score 0 · Answer 3 · answered Oct 16 '13 at 11:20

You can use this structure to do it. It uses ThreadPool so it's not creating a lot of threads, though better idea will be to use APM to send emails, hence no threads will be wasted.

It waits for all background operations are done. The data is your dataset's rows.

int opsLeft = data.Count();
using (var mre = new ManualResetEvent(false))
{
    foreach (DataRow row in data)
    {
        string refId = Convert.ToString(row["ReferenceID"]);
        if (!string.IsNullOrEmpty(refId))
        {
            ThreadPool.QueueUserWorkItem(_ =>
            {
                apeDBAdapter.SendEmail(personId, refId, parentReferenceID, customerName, queueId);
                if (Interlocked.Decrement(ref opsLeft) == 0)
                    mre.Set();
            });
        }
        else
        {
            if (Interlocked.Decrement(ref opsLeft) == 0)
                mre.Set();
        }
    }
    mre.WaitOne();
}

Multithreaded foreach

3 Answers3