Consuming tasks from GAE Task Queue

Question

I would like to use Task Queue with my App in GAE. As I know (according to Task Queue documentation and API), we can add tasks to push queue, and then they are being consumed automatically by the app. Also in pull queue, we can add tasks, and tasks are being consumed automatically according to scaling properties we handle.

I would like to know whether there is ability to consume tasks from task queue (push or pull queues), when we want by calling a consume-like method?

Thanks

score 3 · Accepted Answer · answered Jun 04 '12 at 20:08

3

I think you misunderstood how push and pull task queues works. In both queues you add tasks to the queue, but while in push queue the system will dequeue the task and assign it to the apropriate handler in pull queue you need to dequeue the task(s) using lease_tasks and handle the task.

answered Jun 04 '12 at 20:08

Shay Erlichmen

31,691
7
68
87

Thanks for you answer. Also I would like to know whether it is ok to use Task Queue (pull queue) as a message queue to GAE (Since GAE does not provide a message queue for it). Also if it is not, what is the best option to use as a message queue to GAE? I also referred to http://stackoverflow.com/questions/625146/memcache-based-message-queue-for-app-engine, but according to what I know, it is not possible to connect these memcache based message queues (MemcacheQ, sparrow etc.) to GAE as GAE supports only Java and Python. – Dulini Atapattu Jun 05 '12 at 04:15
Why would it not be okay? And what precisely is the difference between a message queue and a task queue? – Nick Johnson Jun 05 '12 at 05:48
@NickJohnson: As far as I know, the Task queue is also a message oriented middle-ware though not full fledged. I also referred to http://stackoverflow.com/questions/10075817/message-queue-vs-task-queue-difference and your comment there too, and according to all that I have read so far, I think it is ok to use pull queue as a message queue... Thanks – Dulini Atapattu Jun 05 '12 at 16:03
What does it actually mean for something to be "middle-ware though not full fledged"? – Nick Johnson Jun 06 '12 at 02:47
As I read, it seems Task queue is not a full-fledged message oriented middleware. Please I would like to have any clarification regarding this, since I have not used it before. thanks – Dulini Atapattu Jun 06 '12 at 05:26

score 1 · Answer 2 · answered Jun 04 '12 at 20:16

Note that the tasks added to a pull queue are not automatically consumed.

The consume-like method you want is lease_tasks (note this is for pull queues only). If you want to consume the tasks at a given time you can have an endpoint that processes these tasks. Also you can set cron jobs to call that particular endpoint.

Consuming tasks from GAE Task Queue

2 Answers2