Sending mass email SLOW with PHP PEAR Mail and ALT-N Mdaemon pro

Question

We have a growing mailing list which we want to send our newsletter to. At the moment we are sending around 1200 per day, but this will increase quite a bit. I've written a PHP script which runs every half hour to send email from a queue. The problem is that it is very slow (for example to send 106 emails took a total of 74.37 seconds). I had to increase the max execution time to 90 seconds to accomodate this as it was timing out constantly before. I've checked that the queries aren't at fault and it seems to be specifically the sending mail part which is taking so long.

As you can see below I'm using Mail::factory('mail', $params) and the email server is ALT-N Mdaemon pro for Windows hosted on another server. Also, while doing tests I found that none were being delivered to hotmail or yahoo addresses, not even being picked up as junk.

Does anyone have an idea why this might be happening?

    foreach($leads as $k=>$lead){

 $t1->start();

 $job_data = $jobObj->get(array('id'=>$lead['job_id'])); 

 $email = $emailObj->get($job_data['email_id']);


 $message = new Mail_mime();
 //$html = file_get_contents("1032.html");

 //$message->setTXTBody($text);

 $recipient_name = $lead['fname'] . ' ' . $lead['lname'];

 if ($debug){
  $email_address = DEBUG_EXPORT_EMAIL;
 } else {
  $email_address = $lead['email'];
 }
 // Get from job 
 $to   = "$recipient_name <$email_address>";
 //echo $to . " $email_address ".$lead['email']."<br>";

 $message->setHTMLBody($email['content']);

 $options = array();
 $options['head_encoding']  = 'quoted-printable'; 
 $options['text_encoding']  = 'quoted-printable'; 
 $options['html_encoding']  = 'base64'; 
 $options['html_charset']  = 'utf-8'; 
 $options['text_charset']  = 'utf-8'; 

 $body = $message->get($options);

 // Get from email table
 $extraheaders = array(
      "From"  => "Sender <sender@domain.com>", 
      "Subject" => $email['subject']
     );

 $headers = $message->headers($extraheaders);

 $params = array();
 $params["host"]  = "mail.domain.com";
 $params["port"]  = 25;
 $params["auth"]  = false;
 $params["timeout"]  = null;
 $params["debug"]  = true; 

 $smtp = Mail::factory('mail', $params);

 $mail = $smtp->send($to, $headers, $body);



 if (PEAR::isError($mail)) {

    $logObj->insert(array(
   'type'   => 'process_email',
   'message' => 'PEAR Error: '.$mail->getMessage()
   )); 
  $failed++;   
 } else {
  $successful++;
  if (DEBUG) echo("<!-- Message successfully sent! -->");

    // Delete from queue 
    $deleted = $queueObj->deleteById($lead['eq_id']);

    if ($deleted){
     // Add to history
     $history_res = $ehObj->create(array(
        'lead_id' => $lead['lead_id'],
          'job_id' => $lead['job_id']
          )
         );

     if (!$history_res){
    $logObj->insert(array(
     'type'   => 'process_email',
     'message' => 'Error: add to history failed'
     )); 
     }
    } else {
      $logObj->insert(array(
     'type'   => 'process_email',
     'message' => 'Delete from queue failed'
     ));      
    }
 }

 $t1->stop();

}

This doesn't solve your problem, but I would consider hiring a mailing list company to manage your emails. Once you start sending out mass mails yourself, you're bound to make mistakes. Best case your users get poor service, worst case your IP is blacklisted by the mail providers. — Stephen, Dec 13 '10 at 21:31
To reiterate, a company that specializes in mass mailing will be far more efficient and effective than a home grown solution. This will free up your time to work on your core product or assets as well. — Stephen, Dec 13 '10 at 21:33
Could be an option. As we already have a system to manage and queue emails, we'd just be looking to send emails externally. Any service that you would recommend? I had a look at send blaster but it seems to be a replacement for the system we have already. — Greg, Dec 14 '10 at 11:53
http://www.mailchimp.com/ comes to mind. 1k subscribers and 6k emails per month is free. And they have flat rate plans with no sending limits for pretty good pricing. — Stephen, Dec 14 '10 at 15:57

score 0 · Answer 1 · answered Dec 13 '10 at 21:58

0

hard to tell. You should profile your code using xdebug to pintpoint low hanging fruit.

Also I think you might consider using a message queue to process your e-mail(redis/beanstalkd/gearmand/kestrel) asynchronous or using third-party dependency like for example google app engine which is very cheap($0.0001 per recipient/first 1000 emails a day free)/reliable. That will cost you about 10 cent a day when considering your load.

answered Dec 13 '10 at 21:58

Alfred

60,935
33
147
186

Are you referring to something like this? http://code.google.com/appengine/docs/python/mail/sendingmail.html. I already use a custom message queue built in PHP, the queue itself seems fine it's just the sending part. – Greg Dec 14 '10 at 11:50
That was exactly what i was referring too. Maybe profiling will give you a little bit more insight? – Alfred Dec 14 '10 at 12:08

score 0 · Answer 2 · answered Dec 13 '10 at 22:09

you're facing a couple of different problems.

1.) to send a lot of emails you really need a mailer queue and several mail servers to get mail from that queue and process the mail in turn (round robin style <-- this link is related but not perfectly specific to your needs. [It's enough to get you started]).

2.) your mail is likely not getting to Hotmail/yahoo for one of 2 reasons.

a.) you don't have RDNS properly configured and when the look for your IP (from the heder) it is not mapping back right to your domain in the header. or

b.), you've already been flagged as a spammer on SPAMHAUS or whatever the blacklisting service du jour is.

Sending mass email SLOW with PHP PEAR Mail and ALT-N Mdaemon pro

2 Answers2