0

I received the monthly bill today for Amazon SQS and I was surprised to see 600.000 requests usage to my queue.

All I am doing is running one single task every minute. How does this add up to 600.000 requests?

@celery.task(name='tasks.check_for_events')
@periodic_task(run_every=timedelta(minutes=1))  
def check_for_events():    
    now = datetime.utcnow().replace(tzinfo=utc,second=00, microsecond=00)
    events = Event.objects.filter(is_reminder_sent = False).filter(reminder_date_time__range=(now - timedelta(minutes=1), now))    
    dthandler = lambda obj: obj.isoformat() if isinstance(obj, datetime) else None    
    for event in events:
          sendEmail.delay( ...)


@celery.task(name='tasks.sendEmail')
def sendEmail(...)
    ....

I am still new to celery so it might be that I am doing something fundamentally wrong. Any tips please?

Houman
  • 64,245
  • 87
  • 278
  • 460

1 Answers1

4

First, SQS considers sends, receives and deletes as requests. So that's

>>> 600000 / 3
200000

tasks.

Last month there were 31 days. So with the check for events task alone there were

>>> 60 * 24 * 31
44640

tasks or

>>> 44640 * 3
133920

requests used upfront.

Now, you'd only have to average

>>> (200000.0 - 44640) / 44640
3.4802867383512543

events per check_for_events invocation to get to 600k requests.

Now I'm not certain what volume of events you're dealing with, but perhaps this puts the usage more into perspective.

Loren Abrams
  • 1,002
  • 8
  • 9
  • 1
    +1 from me. Thanks for the fantastic breakdown. Since I am very new to messaging, I wasn't aware that 3 messages are sent per task. Hence since 3 actions (delete, sent, receive) are done per task, which runs per minute -> your approximate 3.48 per event seems correct. And I haven't set up anything wrong. Thank you. that was very kind – Houman Feb 04 '13 at 11:09