Stack Overflow
Stack Overflow

Questions tagged [task-queue]

With the Task Queue API, applications can perform work outside of a user request, initiated by a user request.

Tasks queues are an efficient and powerful tool for background processing; they allow your application to define tasks, add them to a queue, and then use the queue to process them in aggregate. You name queues and configure their properties in a configuration file named queue.yaml. A queue can be one of two types—push or pull—based on your needs.

Push queues function only within the App Engine environment. These queues are the best choice for applications whose tasks work only with App Engine tools and services. With push queues, you simply configure a queue and add tasks to it. App Engine handles the rest. Push queues are easier to implement, but are restricted to use within App Engine. For more information about push queues and examples of how to use them, see Using Push Queues.

In summary, push queues allow you to process tasks within App Engine at a steady rate and App Engine scales computing resources according to the number of tasks in your queue.

757 questions
0
votes
1 answer

How to ignore the excess requests in Task Queue?

Assume I have this queue. Update-total-value 30/m 2 If I get a million requests on that second, I just need to process two tasks(One is enough, But just trying to…
Distraction Arrestor
  • 301
  • 3
  • 12
0
votes
4 answers

How to get notified of Google App Engine Task Queue failure

I am using a GAE push Task Queue. I want the task to run once and if it fails I want it to notify me somehow. I also don't want it to do any retries. If it fails I want it to sit in the queue until I rerun it manually. I can manually catch all…
goody44
  • 128
  • 1
  • 7
0
votes
1 answer

Why are my celery worker processes processing everything in serial?

I'm running multiple celery worker processes on a AWS c3.xlarge (4 core machine). There is a "batch" worker process with its --concurrency parameter set to 2, and a "priority" process with its --concurrency parameter set to 1. Both worker processes…
Nathan Breit
  • 1,661
  • 13
  • 33
0
votes
1 answer

Create an automatic periodic job to execute every 10 min on App Engine

My problem is that I want to schedule a job on a server every 10 min which will download, parse and store rss feeds on a database. I have read about Task Queue on App Engine but i am not sure if that would be a right tool for the job?? Thanks…
Adrian L
  • 443
  • 1
  • 8
  • 19
0
votes
1 answer

Running into out of order issue with my task queue (kue.js)

I'm not really sure how to fix this, but it's rather annoying and screwing up things for me in production. I'm using kue.js to complete a task after a user has done something. The most typical case on my site is for uploading images and sending…
bob_cobb
  • 2,229
  • 11
  • 49
  • 109
0
votes
0 answers

Google app engine did not run one of the tasks in the push queue

I have a B2 machine running tasks from the default push queue. I added 4 tasks to the queue, and task2 didnt run, no logs about it appear in the admin console, and i dont see any failures. task1, task3, and task4 successfuly finished. the way i…
adi ohaion
  • 374
  • 6
  • 21
0
votes
1 answer

Define Task Queue Handler in Google Endpoint

I am using Google Endpoint in Python. Now I want to add a push queue but I do not want anyone has the right to trigger this queue. For example, I create a queue when user call one…
YungChun
  • 105
  • 5
0
votes
1 answer

Google App Engine: Modifying 1000 entities using TaskQueue

I am hoping to modify 1000 entities using task queue, as suggested Zig Mandel in my original question here: Google App Engine: Modifying 1000 entities I have a UserAccount kind like this: class UserAccount(ndb.Model): email =…
hyang123
  • 1,208
  • 1
  • 13
  • 32
0
votes
2 answers

Simple queue system with scheduling support

I would need a queue system with scheduling support for a Ruby on Rails application. This is scheduling a job to be run at a certain time. Also it could be modified to be run sooner or later, or removed from the queue. I'm thinking of using Sidekiq,…
David Morales
  • 17,816
  • 12
  • 77
  • 105
0
votes
1 answer

GAE (go) push queues rate not enforced

I am trying to have some task in a push queue to be executed at rate of about once a second. Here is how I defined my queue.yaml queue: - name: default rate: 1/s bucket_size: 1 max_concurrent_requests: 1 I have a handler (for…
Itay Karo
  • 17,924
  • 4
  • 40
  • 58
0
votes
1 answer

How to stop appengine from creating automated multiple default queue?

I have not written any code which creates taskqueue in application. still while running application multiple queues are generated in backgraound. This application is Multitenant application and NamespaceManager.set() is called before performing DAO…
priyanka_rao
  • 465
  • 1
  • 4
  • 20
0
votes
1 answer

Google appengine creates deferred queue(/_ah/queue/__deferred__) for all request in multitenancy application

Appengine creates deferred queue(/_ah/queue/deferred) for every hit or every request on Multitenant applications site, hence creating and running alot of queues which gives wrong results as well as excessive use of queue leading to exceeded…
priyanka_rao
  • 465
  • 1
  • 4
  • 20
0
votes
1 answer

pulling current date queue

I have a view that lists employee (EmpID), request number (ReqNo), date request was opened (OpenDate) and the date it was moved to the next step in the process (AssignDate). What I am trying to do is get an average of the daily queue size. If EmpID…
MKowalski
  • 37
  • 2
  • 7
0
votes
2 answers

Execute tasks in a particular order

I have this loop: listGames = [] for home in range(totalPlayers - 1): for away in range(home + 1, totalPlayers): listGames.append((home, away)) print listGames match_num = 1 for game in listGames: player1 = listPlayers[game[0]] …
LordAro
  • 1,269
  • 3
  • 18
  • 35
0
votes
1 answer

/mapreduce/workerCallback producing http 429 response

I'm working in Java and was able to kick-off a mapreduce job. The job made it through the ShardedJob stage, but is now stuck on the ExamineStatusAndReturnResult stage. In the task queue I see a number of jobs like:…
wspeirs
  • 1,321
  • 2
  • 11
  • 22
1 2 3
…
50 51