How to ignore the excess requests in Task Queue?

Question

Assume I have this queue.

  <queue>
    <name>Update-total-value</name>
    <rate>30/m</rate>
    <bucket-size>2</bucket-size>
  </queue>

If I get a million requests on that second, I just need to process two tasks(One is enough, But just trying to be in safer side) and ignore everything else.

I understood that if I implement the above queue, It processes 2 tasks first and 30 for every minute! And no task will be lost. My requirement is to process the first two and lose all other tasks.

Please advise how to achieve this?

Thanks, Karthick.

Answer 1

Number one: you should control all code that is making task queue requests. In order to even make a request on that task queue route successfully, the call needs to generate from inside your code. Thus, you should be able to rate-limit events, check queues, etc. so that you don't see more requests than you need. If it's a resource contention between multiple users, implement a lock using memcache, Datastore, or in memory of your instances (depending on your scaling options and communication between instances).

Number two: the documentation on deleting tasks is part of the Task Queue Java API docs. As you can read there, from your own code (where such a decision should be made) you have the option to delete individual tasks, or to purge an entire queue. You could use your imagination of which will be better for you. Notice that tasks will continue to execute while this operation is in progress (purge), so in conclusion, you should either rate limit better or make sure that the task queue operations are idempotent.