Stack Overflow
Stack Overflow

Questions tagged [task-queue]

With the Task Queue API, applications can perform work outside of a user request, initiated by a user request.

Tasks queues are an efficient and powerful tool for background processing; they allow your application to define tasks, add them to a queue, and then use the queue to process them in aggregate. You name queues and configure their properties in a configuration file named queue.yaml. A queue can be one of two types—push or pull—based on your needs.

Push queues function only within the App Engine environment. These queues are the best choice for applications whose tasks work only with App Engine tools and services. With push queues, you simply configure a queue and add tasks to it. App Engine handles the rest. Push queues are easier to implement, but are restricted to use within App Engine. For more information about push queues and examples of how to use them, see Using Push Queues.

In summary, push queues allow you to process tasks within App Engine at a steady rate and App Engine scales computing resources according to the number of tasks in your queue.

757 questions
0
votes
1 answer

How can an App Engine pull queue be simultaneously empty and yet still have an oldest task?

I'm confused as to whether my queues are truly empty. From this view, the queue called "squid-pri-0" says it has 14 tasks in the queue: but when I click on it, it says it's empty (see below). However, I also see an "old task" that is not blank,…
ryan
  • 2,311
  • 3
  • 22
  • 28
0
votes
1 answer

GAE python: Get and post request working without submitting a post request through a form

This is a continuation of this question What follows is an abbreviated version of the original code. I tried to include the most relevant parts and left out the part of the script used by a cron job which updates the Datastore with values. Then, in…
655321
  • 411
  • 4
  • 26
0
votes
2 answers

How can I make resque worker process other jobs while current job is sleeping?

Each task I have work in a short bursts, then sleep for about an hour and then work again and so on until the job is done. Some jobs may take about 10 hours to complete and there is nothing I can do about it. What bothers me is that while job is…
Andrew
  • 8,330
  • 11
  • 45
  • 78
0
votes
1 answer

Getting a pull task queue to work with cron

I'm new to python and GAE and I've been trying to learn how to work with tasks queues, cron, and datastore for my own project through examples but haven't had much luck at all. I trying to modify the Votelator code from this Google Developers…
655321
  • 411
  • 4
  • 26
0
votes
2 answers

PHP Gearman Duplicate tasks

I got: Debian 6 nginx 1.2.6 PHP 5.3.20-1~dotdeb.0 (fpm-fcgi) (built: Dec 24 2012 11:53:16) gearmand 1.1.4 pecl gearman ext 1.1.1 And I got two scripts (Probably I copied them from php.). Client:
ns3777k
  • 41
  • 1
  • 5
0
votes
1 answer

dynamically load-balanced task queue

Set up Front-end machine receives web requests and puts 3 different kinds of tasks into Gearman server. 3 worker machines registered with that gearman server, each can perform any of those tasks. Different tasks consume different resources. Each…
Alex
  • 11,479
  • 6
  • 28
  • 50
0
votes
4 answers

Find minimum queue size among threads

I am trying to implement a new scheduling technique with Multithreads. Each Thread has it own private local queue. The idea is, each time the task is created from the program thread, it should search the minimum queue sizes ( a queue with less…
aram
  • 71
  • 1
  • 10
0
votes
1 answer

GAE tasks scheduled for backend execution returning 405 error

I'm having trouble understanding why frontend tasks execute correctly but backend tasks are returning 405 error. The following code: queue.add(withUrl(url).taskName(taskName).method(Method.GET)); works without a hitch as it is being processed on the…
klactose
  • 1,190
  • 2
  • 9
  • 26
0
votes
2 answers

Scheduling tasks in multithreads

I am trying to schedule tasks in multi threaded systems. my idea is to have a local queue per thread, each thread will fetch the job from its local queue. But when the thread reaches some threshold, it should not fetch the job, rather it should…
Rd7
  • 45
  • 2
  • 9
0
votes
1 answer

Measuring Task Queue Costs in Google App Engine

I am measuring the cost of requests to GAE by inspecting the x-appengine-estimated-cpm-us-dollars header. This works great and in combination with x-appengine-resource-usage and x-traceurl I can even get more detailed information. However, a large…
Ingo
  • 1,552
  • 10
  • 31
0
votes
1 answer

Queue does not finish after tasks are done

This is a script I'm using to test the Queue module: from Queue import Queue from threading import Thread def worker(fruit_queue): while True: fruit = fruit_queue.get() print 'A fruit: ' + fruit …
user1447941
  • 3,675
  • 10
  • 29
  • 34
0
votes
1 answer

Retaining task queues on deploying a new version

For Google App Engine apps, are task queues cleared when you deploy a new version of your app? If the queues are cleared, what's the best strategy for persisting tasks across versions?
vaughandroid
  • 4,315
  • 1
  • 27
  • 33
0
votes
2 answers

Error (401 Unauthorized) accessing GAE Task Queue (pull queue)

I'm getting this error below trying to access a pull task queue. Until the last friday (Oct, 5) it was working perfectly. Today (Oct, 8) it wasn't working. I run it from a Websphere Application Server v7 with the 2 imported certificates below:…
user1633962
  • 21
  • 2
0
votes
2 answers

App Engine Backend Quota Not Touched

I have a dynamic backend process that gets called from within my frontend app code. To do this it must be visible via app.yaml. But when the backend gets run in this way I see no change in my backend quota on the admin panel, leading me to believe…
user1561108
  • 2,666
  • 9
  • 44
  • 69
0
votes
1 answer

Unable to connect to MongoDB server at Interrupted system call

I'm running a Rackspace cloud server CentOs + apache2 + php 5.4 + pcntl module with a basic Kohana php framework with a mongoDb task module that forks children processes. I get the following error if I try to run more then 1 child task process at…
JonnyJon
  • 1,215
  • 2
  • 9
  • 6
1 2 3
…
50 51