3

I hope you guys will bear with me for my total lack of direction in case of Threading. I have to implement a Mail Queue Processing System where I have to send emails queued up in a database through a Windows Service.

It is not a producer-consumer pattern. I have fetched say 10 rows at a time into a datable. The datatable contains the serialized MailMessage object and SMTP sending details. If I have to use say a fixed number of threads (say 6 threads) Now how would I go about fetching a row from the datatable in a thread and then send the mail and again return to see if any more rows are remaining?

Any simple logic to implement this will do, preferably with a simple example in C#.

i am using .NET 3.5

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
Pankaj Kumar
  • 1,748
  • 6
  • 28
  • 41
  • What is the reason for those multiple threads? The fetching of the rows or the sending of the mails? – Daniel Hilgarth Feb 24 '11 at 08:38
  • The mails need to be sent in bulk for newsletters and invitations.There is a component named ASPEmail that achieves fast email sending using simultaneous sessions. I want to try multithreading if it helps in improving performance. – Pankaj Kumar Feb 24 '11 at 09:13

2 Answers2

6

Since sending emails is an I/O bound process, so spawning threads to send emails won't achieve much (if any) speedup.

If you're using the SMTP server that's part of Windows then when you "send" an email it doesn't actually get sent at that instant. It sits in a queue on the server and the server sends them as fast as it can. Sending emails is actually a slow process.

I guess what I'm saying is there are two options:

  1. Just send them sequentially and see if that meets your performance requirements.
  2. You could use a parallel programming concept called "Data Parallel" I've exampled it with examples in an blog post Data Parallel – Parallel Programming in C#/.NET

Basically, what you're doing is, you'll get all of your data (in one go). The reason is getting data in batches will also slow your process down so if you're interested in performance (which is why I'm guessing you're attempting to use threads), then don't do multiple rounds trips to the database server (which is also I/O bound on two levels, network I/O as well as disk I/O).

So get your data, and split it into chunks or partitions. This is all explained in the article I pointed to. The naive implementation would be the number of chunks equals the number of cores on the machine.

Each chunck is processed by one thread. When all threads are done, you're done. Withthe new features of the ThreadPool in .NET 4.0 (if you use Parallel.For or PLINQ or Tasks) you'll get some other benefits such as "work stealing" to further speed up the work.

Parallel.For/Parallel.ForEach will work well for you I'd think.

EDIT

Just noticed the .NET 3.5 requirement. Well the concepts still apply, but you don't have Parallel.For/ForEach. So here is an implementation (modified from my blog post) that uses the ThreadPool and using a Data Parallel technique.

    private static void SendEmailsUsingThreadPool(List<Recipient> recipients)
    {
      var coreCount = Environment.ProcessorCount;
      var itemCount = recipients.Count;
      var batchSize = itemCount / coreCount;

      var pending = coreCount;
      using (var mre = new ManualResetEvent(false))
      {
        for (int batchCount = 0; batchCount < coreCount; batchCount++)
        {
          var lower = batchCount * batchSize;
          var upper = (batchCount == coreCount - 1) ? itemCount : lower + batchSize;
          ThreadPool.QueueUserWorkItem(st =>
          {
            for (int i = lower; i < upper; i++)
              SendEmail(recipients[i]);
            if (Interlocked.Decrement(ref pending) == 0)
              mre.Set();
          });
        }
        mre.WaitOne();
      }      
    }

    private static void SendEmail(Recipient recipient)
    {
      //Send your Emails here
    }
  }

  class Recipient
  {
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string EmailAddress { get; set; }
  }

So, get your data and call SendEmailUsingThreadPool() passing it your data. Of course don't call your method that :). If you have a DataSet/DataTable then simply modify the implementation to accept a DataSet/DataTable. This methods takes care of partioning your data into chunks so you don't have to worry about any of that. Simply call it.

Shiv Kumar
  • 9,599
  • 2
  • 36
  • 38
  • thanks for a detailed answer.The main reason to implement such a system was when say 5000 users were sent mail using SmtpClient then the code would block for a very long time. Since now there are being stored in a DB it will be much faster. The threads are desired to process the queue quickly and pass the responsiblity to the SMTP server. – Pankaj Kumar Feb 24 '11 at 10:23
  • +1 for the .Net 3.5 implementation. I will try that and get back to you. – Pankaj Kumar Feb 24 '11 at 10:26
  • @Pankaj, I'm not sure I follow the first part of your comment. FYI: You'll still use SmtpClient to send emails. I'm assuming when you say the code was blocked, you means that in a ASP.NET application (the main app) the code was blocked so you're moving the sending to a Windows service? – Shiv Kumar Feb 24 '11 at 10:30
  • @Pankaj, seeing that you're sending emails at some interval (rather than instantly), you want to have a regular console application to do this and you can then have it run at regular times or on a specific day of week or month using the Task Scheduler. It's easier to debug and you can always convert it to a windows service after you've had a few successful runs in production, though there is no real value (in this case) to having it run as a service. – Shiv Kumar Feb 24 '11 at 10:38
0

You need to fetch messages in to memory in one place, and then route them to separate threads. I guess

gabba
  • 2,815
  • 2
  • 27
  • 48