How to properly throttle message sending with nodemailer SES transport?

Question

If you use rate or connection limiting then you can also use helper methods to detect if the sending queue is full or not. This would help to avoid buffering up too many messages.

It also provides an example:

let transporter = nodemailer.createTransport({
    SES: new aws.SES({
        apiVersion: '2010-12-01'
    }),
    sendingRate: 1 // max 1 messages/second
});

// Push next messages to Nodemailer
transporter.on('idle', () => {
    while (transporter.isIdle()) {
        transporter.sendMail(...);
    }
});

Unfortunately, this is rather cryptic to me. Does sendingRate: 1 only provides a helper, or does it handle throttling ?

Also this piece of code looks to me like it would loop infinitely as soon as sendMail(...) is executed. Am I missing something here ?

Is there any example or recommendation on how to use this feature ?

Thanks a lot !

I'm sending thousands of email instantly. The rate limit is fixed to 5 but I'm not checking the `isIdle()` state. I'm getting throttling error. Don't know why? — Sowmay Jain, Jul 19 '18 at 13:49

score 2 · Answer 1 · answered Sep 25 '19 at 16:59

2

From the documentation:

SES can tolerate short spikes but you can’t really flush all your emails at once and expect these to be delivered. To overcome this you can set a rate limiting value and let Nodemailer handle everything – if too many messages are being delivered then Nodemailer buffers these until there is an opportunity to do the actual delivery.

I don't think listening for the idle event is mandatory, it's only needed if you want to avoid Nodemailer buffering messages. I have an SES send rate of 15 messages per second and regularly throw 250 emails at once at Nodemailer and don't hit any throttling issues.

answered Sep 25 '19 at 16:59

Dougc

843
11
20

2

Are you taking a list of emails and mapping over them while calling `transporter.sendMail()` to create a list of promises? Then doing a `Priomise.all()`? I am trying to figure this out myself right now, where I want to send an email to over 500 destinations but not use CC or BCC. My SES rate is 14/sec so did not know if I was required to do batched of 14 or could use the 50 limit as stated for SES. Or simply rely on nodemailer to handle the throttling. – natac Jan 08 '20 at 11:54
@natac any update/solution regarding your particular case mentioned? – Prem popatia Sep 20 '20 at 13:11
1

@Prempopatia Well I have a working solution. However I had to move away from nodemailer due to my lambdas not having internet access inside the VPC which is peered to MongoDB Atlas. Now what I do is send all the info required for emails to SQS. I have a lambda which is polling the queue and takes in batches of 10. I make sure this lambda can only scale so many. That way I get my throttling with the lambda. I send off the 10 email messages coming from sqs and check elapsed time. I doubt it is the best solution but I did not receive any advice from AWS dispite having business support.... sad. – natac Sep 20 '20 at 20:57

Sunny · Answer 2 · 2018-04-11T16:14:51.897

1

You are right, the while loop only appears to be there for testing sending rate. Once you remove the while loop the code in documentation should work fine.

transporter.on('idle', () => {
    transporter.sendMail(...);
});

edited Apr 11 '18 at 16:14

answered Apr 10 '18 at 23:00

Sunny

11
3

I'm sending thousands of email instantly. The rate limit is fixed to 5 but I'm not checking the `isIdle()` state. I'm getting throttling error. Don't know why? – Sowmay Jain Jul 19 '18 at 13:49

score 0 · Answer 3 · answered Mar 27 '21 at 14:32

You don't need the while loop or the on idle handler. Just set the sendingRate and then use sendMail as normal.

transporter = nodemailer.createTransport({
  SES: { ses, aws },
  sendingRate: 14,
});

const params = {
  from: 'EMAIL',
  to: 'EMAIL',
  subject: 'Message',
  html: 'I hope this <b>message</b> gets sent!',
  text: 'I hope this message gets sent!',
  // attachments: [{ filename: 'card.pdf', content: data, contentType: 'application/pdf' }],
};

transporter.sendMail(params, (err, info) => {
  if (err) {
    console.log(JSON.stringify(err));
  }
  console.log(info.envelope);
  console.log(info.messageId);
});

Important thing to note here, nodemailer waits for the next second to continue with the next batch of throttled emails and the next and so on. So if you are running a script that exits immediately after calling the last sendMail(), the throttled emails will never get sent. Make sure the process runs until all emails are sent by listening to on idle or use settimeout.

How to properly throttle message sending with nodemailer SES transport?

3 Answers3