Most effective way to poll an Amazon SQS queue using Node

Question

My question is short, but I think is interesting:

I've a queue from Amazon SQS service, and I'm polling the queue every second. When there's a message I process the message and after processing, go back to polling the queue.

Is there a better way for this?, some sort of trigger? or which approach will be the best in your opinion, and why.

Thanks!

score 29 · Answer 1 · answered Nov 16 '16 at 18:33

A useful and easily to use library for consuming messages from SQS is sqs-consumer

const Consumer = require('sqs-consumer');

const app = Consumer.create({
  queueUrl: 'https://sqs.eu-west-1.amazonaws.com/account-id/queue-name',
  handleMessage: (message, done) => {
    console.log('Processing message: ', message);
    done();
  }
});

app.on('error', (err) => {
  console.log(err.message);
});

app.start();

It's well documented if you need more information. You can find the docs at: https://github.com/bbc/sqs-consumer

The first line should now be: `const { Consumer } = require('sqs-consumer');` — Falcon Stakepool, Mar 28 '23 at 05:10

score 3 · Answer 2 · answered Jul 08 '15 at 21:19

3

yes there is: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html

you can configure the SQS queues to have a "receive message wait time" and do long polling.

so you can set it to say 10 seconds, and the call will come back only if you have a message or after the 10 sec timeout expires. you can continuously poll the queue in this scenario.

answered Jul 08 '15 at 21:19

Mircea

10,216
2
30
46

1

But once all the messages have been consumed, won't long polling just stop, just like short polling does. You will have to actually manually "pull" again to get the next lot. – Epirocks Jun 28 '21 at 20:14
1

no. the long polling does not stop. it will still poll with a timeout. so the polling effectively blocks until the timeout expires or you have a message. – Mircea Aug 14 '21 at 23:46
I assume the question is, what's best practice or best pattern to maintain consistent polling over time, so that, for instance, if you're using long polling with a 20 second timeout, you're consistently starting a new poll the moment the old poll ends (either with messages or not). At least, that's the question I'm trying to answer. I can throw something together that'll get the job done, just want to make sure I'm doing it "the right way" if there is one. – Jason Aug 23 '23 at 15:17

score 0 · Answer 3 · answered Jul 09 '15 at 07:35

As mentioned by Mircea, long polling is one option.

When you ask for a 'trigger', I believe you are looking for something other than continuously polling SQS on your own. If that is the case, I'd suggest you look at AWS Lambda. It allows you to put code in the cloud, which automatically gets triggered on your configured events, such as SNS event, a file pushed to S3 etc.

http://aws.amazon.com/lambda/

Most effective way to poll an Amazon SQS queue using Node

3 Answers3