5

I'm trying to implement swiftmailer into this mailing system. my client has about 300k active emails that need to be sent on a semi regular basis. the system was originally configured for sendmail and php's mail() function. I've since installed the latest version of postfix.

it may be my expectations were way too high but i was under the impression this thing can put a lot of emails into the queue FAST, which is exactly what i need. all the rate handling and throttling is done on the postfix side, so being able to queue them as fast as my postfix settings can handle would be great.

while i could implement methods to insert the contact directly into the queue, i'd rather limit the input of emails going into the queue based on various options such as global send rates for the smtp server.

the code below is just something basic to test. it loops through 30 separate mailing smtp accounts, each with it's own rate properties. i'm trying to pull the max amount of emails out of the db per cron, then send then all using batchsend(), then loop to the next smtp account, send max, etc..

technically it does work, however it's really slow. at a rate of 60/min per smtp account, it takes about 15-20 seconds each which obviously will not work and not at all what i had expected.

is there something really obvious about why this would send so slow? the smtp servers appear to be fine, no overload or anything like that. no postfix errors, nothing obvious.

once the emails make it to the queue, i let postfix work it's magic. it's getting it into the queue at a reasonable rate that's become difficult. i know swiftmailer is a magic solution to all my problems, but i'm sure it should be sending quicker than it is. any ideas or suggestions?

$query = "SELECT * FROM `smtp`";
    $result = mysql_query($query) or die(mysql_error());
    $num_rows1 = mysql_num_rows($result);
    while($row = mysql_fetch_array($result)){
        $smtp = $row['ip'];
        $login = $row['user'];
        $pass = $row['pass'];
        $smtp_domain = $row['domain'];   

    $querya = "SELECT * FROM `mailer_lists` ml JOIN `mailer_controller` mc ON ml.project_name = mc.project_name LIMIT ".$global_limit."";
    $resulta = mysql_query($querya) or die(mysql_error());
    $num_rows2 = mysql_num_rows($resulta);

    // quickly check if any mail returned query
    if ($num_rows2 < 1){
        mysql_close($connection);
        echo "Nothing to mail... \n";
        die();
        }

    while($rowa = mysql_fetch_array($resulta)){
        $email[] = $rowa['email'];
        $project_name = $rowa['project_name'];
        $from_name = $rowa['from_name'];
        $subject = $rowa['subject'];
        $body = $rowa['body'];
        $from = array($spun_from_email => $spun_from_name);
    }    

    require_once 'swift/swift_required.php';
        $transport = Swift_SmtpTransport::newInstance(''.$smtp.'', 25)
          ->setUsername($login)
          ->setPassword($pass)
          ;
        $mailer = Swift_Mailer::newInstance($transport);
        $message = Swift_Message::newInstance()
          ->setSubject($subject)
          ->setFrom($from)
         // ->addPart('add part message', 'text/html')
         // ->attach(Swift_Attachment::fromPath(''))
          ->setTo($email)
          ->setBody($body)
        ;
        $mailout = $mailer->batchSend($message);
         // ->addPart('add part message', 'text/html')
         // ->attach(Swift_Attachment::fromPath(''))
          ;  


            $queryb = "DELETE FROM `mailer_lists` WHERE `project_name` = '".$project_name."' AND `email` = '".implode('\' OR `email` = \'',$email)."'";  
            $resultb = mysql_query($queryb) or die(mysql_error());
            $c++;
            echo "sent $num_rows1 emails to smtp $c \n";
        }
Robin
  • 9,415
  • 3
  • 34
  • 45
john
  • 1,330
  • 3
  • 20
  • 34
  • Insert some calls to time() around some of your functions and then compare them at the end to see where your bottleneck truly lies. It may not be where you expect – gnxtech3 May 23 '11 at 20:51
  • actually it appears my perception of swift mailer was wrong. using batchsend() simplifies the send() method by doing the loop for you, but not necessarily faster. – john May 24 '11 at 01:42
  • Facebooks xhprof sounds like the perfect tool for your problem. It's a very lightweight profiler that can visualize slow downs – David May 24 '11 at 17:30

1 Answers1

4

Your problem is not the library you are using. This is being slow because you are sending a big chunk of data to the email server...so postfix is trying to process all before replying you. Try sending ONE email at the time without closing the connection, so this way the email server can process faster each email you send to it and return an answer faster too.

  • move the following line to the top of the script

require_once 'swift/swift_required.php';

  • Make the connection to the smtp server lazy. You don't need to connect every time you want to send an email. Connect once at the top and check if the connection is alive (otherwise connect again), send the email and clear the fields to, from, etc...

  • grab your smtp settings from the database at the top then remove the giant while. That doesn't make the code readable at all

Gabriel Sosa
  • 7,897
  • 4
  • 38
  • 48
  • Can't agree on this answer. In my ticket system I send mail once with simple body and it is still doing it awfully long. – Eugene Sep 25 '12 at 09:00
  • @Eugene there could be many factors. Where is your relay server? over the same network or are you using some PAAS service like Sengrid or so? Taking that in consideration we can move forward – Gabriel Sosa Sep 26 '12 at 13:11
  • any extract of code you can share? so we can nail down this issue? – Gabriel Sosa Sep 28 '12 at 02:20
  • That would be the code I use http://pastebin.com/VfbyKHLM. I must say right away, that Twig template is not the timing issue here and it is also as light as html file can be. – Eugene Sep 28 '12 at 08:21
  • I doubt you are in the same network unless your code is hosted on google's servers. Connecting to google's smtp over TLS isn't trivial, also you are crossing all internet. Do you have any server over the same network to test? – Gabriel Sosa Sep 28 '12 at 14:04
  • Sorry. Explained myself wrong. No, no other server at this time. but you are saying, that this happens probably to a large scale distance. So it would be normal delay, right? – Eugene Sep 28 '12 at 14:12
  • well, not sending using a server in the same net adds a lot of overhead + plus the TLS encryption. just for a sake of test try using a server in the same net and benchmark – Gabriel Sosa Sep 28 '12 at 14:16