Stack Overflow
Stack Overflow

Questions tagged [task-queue]

With the Task Queue API, applications can perform work outside of a user request, initiated by a user request.

Tasks queues are an efficient and powerful tool for background processing; they allow your application to define tasks, add them to a queue, and then use the queue to process them in aggregate. You name queues and configure their properties in a configuration file named queue.yaml. A queue can be one of two types—push or pull—based on your needs.

Push queues function only within the App Engine environment. These queues are the best choice for applications whose tasks work only with App Engine tools and services. With push queues, you simply configure a queue and add tasks to it. App Engine handles the rest. Push queues are easier to implement, but are restricted to use within App Engine. For more information about push queues and examples of how to use them, see Using Push Queues.

In summary, push queues allow you to process tasks within App Engine at a steady rate and App Engine scales computing resources according to the number of tasks in your queue.

757 questions
0
votes
2 answers

Should I use a Task Queue to add Documents to index on AppEngine

The usual story: I have a com.google.appengine.api.search.Index backed by my datastore. For each document, the documentId is the key of a certain entity. (For visualization, imagine I have a “table” of products and each product has a number of…
Katedral Pillon
  • 14,534
  • 25
  • 99
  • 199
0
votes
0 answers

Django celery - endless task after task, push task to front on user request

I want to use django celery as a bot of sorts that continuously loops through objects and gets data for them. For example, lets say I have a huge database of movies. I want to loop through all the movies, one by one, and get their metadata. If a…
takeshibaconsuzuki
  • 67
  • 1
  • 8
0
votes
1 answer

Sporadic Error in GAE using deferred.defer - RequestTooLargeError: The request to API call datastore_v3.Put() was too large

It looks like this question has been asked many times, and I'm aware that the datastore has a storage limit of 1MB per entity, as explained here. But. I'm still stuck... Here the code I use to defer tasks (largely taken from the Google's sample…
user2779653
  • 918
  • 1
  • 9
  • 26
0
votes
2 answers

Google App Engine Task Queue error

I'm developing a matchmaking system with gae and python. I was finding a consistent system for automatic matchmaking, and I found the task queue. I implemented a cron job to run every 10 minutes to add a queue. However I'm getting the following…
Dashadower
  • 632
  • 1
  • 6
  • 20
0
votes
0 answers

Java producer consumer stock exchange low level

I need to write a server for stock exchange in Java, the server will get many requests from couple of servers per second, my goal is to write this server and find out how many HTTP GET requests it can handle after executing couple of conditions and…
0xIddo
  • 1
  • 2
0
votes
3 answers

Creating a python priority Queue

I would like to build a priority queue in python in which the queue contains different dictionaries with their priority numbers. So when a "get function" is called, the dictionary with the highest priority(lowest number) will be pulled out of the…
sista_melody
  • 49
  • 1
  • 5
0
votes
2 answers

"_countdown" in "deffered.defer" is not working with google app engine

I am trying to run simple function and limit retry using deffered lib. This is the simple use of deferred.defer(Simple_function, para1, para2, _countdown=30). If this function fails the process is retrying above 30 sec.
ZApatel
  • 85
  • 9
0
votes
1 answer

If Task queue failed at some queue then how can i inform to client some queue are failed

Hi i am using task queue for saving customer in customer entity. I am saving 10 customer into the customer entity using task queue. So Task in queue there are 10 customer. if due to some wrong inputs some 5 customer not saved in entity and 5…
Vijay Chougule
  • 123
  • 14
0
votes
1 answer

How to compute the percentage of failed tasks in Google AppEngine?

I am using push task queue from GAE (python). There are times when after X minutes, Y% of the tasks have failed. For this situation, I want to purge the task queue (there is no need to execute them, eventually many will fail). I can configure for a…
octavian1001
  • 314
  • 2
  • 7
0
votes
2 answers

How to call one servlet to another servlet in GWT Using Java

1) Hi I am new gwt. I want to execute one servlet to another servlet. for example i want to execute servlet2 from servlet1 one. I can execute sevlet1 using RPC call so from servlet1 i want to execute servlet2 which have doPost method i want to…
Vijay Chougule
  • 123
  • 14
0
votes
0 answers

How to properly configure task queue to run 30 tasks per hour?

I should run only 30 tasks per hour and maximum 2 tasks per minute. I am not sure how to configure these 2 conditions at the same time. Currently I have the following setup: - name: task-queue rate: 2/m # 2 tasks per minute bucket_size: 1 …
LA_
  • 19,823
  • 58
  • 172
  • 308
0
votes
0 answers

How to handle Laravel Queues when system fall down

I'm using the Laravel 4.2 Queues service (http://laravel.com/docs/4.2/queues) for handling long-time ops. This helps me to handle the requests to server more quickly. But how I can handle the next situation? The server puts the job to the queue and…
joanlopez
  • 579
  • 1
  • 6
  • 17
0
votes
1 answer

App Engine: What are the Default Retry Parameters for Failed Taskqueues

Google App Engine has a robust set parameters to automatically retry failed tasks in the distributed tasksqueue. While the documentation explains what these parameters do, it fails to specify what the default parameters are. What are the defaults…
speedplane
  • 15,673
  • 16
  • 86
  • 138
0
votes
1 answer

How to catch task failure on Google App Engine?

Google App Engine provides us with task queues (task queue reference). I use a push queue configured to retry failed tasks 4 times. I would like to handle a case when task fails completely, all 4 tries haven't succeed. Is there is a way to handle…
Vladyslav Lubenskyi
  • 527
  • 1
  • 7
  • 17
0
votes
0 answers

Deferred Task Request deadline exceeded but work never started

I have a task queue that handles deferred tasks that need to be processed in real time (I have heard this isn't a good idea but can't find any supporting documentation on this.). This queue is hit with an large influx of tasks (2500/min) and then…
David Ward
  • 493
  • 3
  • 15
1 2 3
…
50 51