Stack Overflow
Stack Overflow

Questions tagged [task-queue]

With the Task Queue API, applications can perform work outside of a user request, initiated by a user request.

Tasks queues are an efficient and powerful tool for background processing; they allow your application to define tasks, add them to a queue, and then use the queue to process them in aggregate. You name queues and configure their properties in a configuration file named queue.yaml. A queue can be one of two types—push or pull—based on your needs.

Push queues function only within the App Engine environment. These queues are the best choice for applications whose tasks work only with App Engine tools and services. With push queues, you simply configure a queue and add tasks to it. App Engine handles the rest. Push queues are easier to implement, but are restricted to use within App Engine. For more information about push queues and examples of how to use them, see Using Push Queues.

In summary, push queues allow you to process tasks within App Engine at a steady rate and App Engine scales computing resources according to the number of tasks in your queue.

757 questions
0
votes
0 answers

Make Import compatible with 3rd party API

I am using an API that allows our app to make 600 calls to it per a minute. An import will allow a user to make as many calls to the API necessary to import all of their contacts. This is being tested with databases of up to 25,000 contacts. With…
Colin Rickels
  • 109
  • 7
0
votes
1 answer

How to run separate instance of the app engine application for every user?

I have an application that does a long running job and pushes the task in the task queue. Currently, when different users login to the application and start the upload job, the job merges with the existing task and expected output is not…
Karan Tongay
  • 197
  • 1
  • 17
0
votes
1 answer

Django: conflicting models in (third-party) application

I integrated a third party app into my Django project, and only when I import it will I get this error message. RuntimeError: Conflicting 'task' models in application 'django_q': and . I'm…
Ehvince
  • 17,274
  • 7
  • 58
  • 79
0
votes
1 answer

How do I create a short-lived, single task Google Compute Engine instance?

Question: How to create a lightweight on-demand instance, preconfigured w/ Java8 and my code, pull a task from a task queue, execute the memory-intensive tasks, and shut itself down. (on-demand, high memory, medium cpu, single task…
Benjamin H
  • 5,164
  • 6
  • 34
  • 42
0
votes
1 answer

Should the input options be included with the results in a job/task queue?

I'm working on developing a job queue type system where Jobs can be added to a central queue. Processor then read jobs of the queue, run them, and store the results somewhere. Each Job contains a list of tasks to perform in order and the options to…
Eric Anastas
  • 21,675
  • 38
  • 142
  • 236
0
votes
1 answer

GAE Task queue is keeping negative tasks running in the "Tasks Running" section in the admin console

I am currently working on a Python project with GAE, and as strange as it may seem, my default queue is keeping a negative count of tasks running even though i manually deleted all tasks. Why is it doing this odd thing? and how to stop it? attached…
John
  • 13
  • 5
0
votes
1 answer

Distributed computing on non-equal servers

I have set of tasks and each of them requires some certain condition to be satisfied. For example task needs 100 Mb of RAM to be executed. Also I have multiple servers where workers are placed. Each worker has some configuration that describes how…
Nikita Lapkov
  • 121
  • 1
  • 6
0
votes
1 answer

How can I schedule or queue api calls to maintain rate limit?

I am trying to continuously crawl a large amount of information from a site using the REST api they provide. I have following constraints- Stay within api limit (5 calls/sec) Utilising the full limit (making exactly 5 calls per second, 5*60 calls…
AsifM
  • 680
  • 9
  • 21
0
votes
0 answers

GAE task queue enforced rate throttling performance

I am currently working on building a load testing platform with Google App Engine. My eventual goal is to simulate 1 million users sending data to another GAE server application every 10 minutes. My current implementation is using task queues,…
DShurbert
  • 1
0
votes
1 answer

Android - how to check if ThreadPoolExecutor has reached the queue limit?

I use many AsyncTask classes in my app. I always start them using: if( Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB ) { asyncTask.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR); } else { asyncTask.execute(); } Recently I got a…
Anonymous
  • 4,470
  • 3
  • 36
  • 67
0
votes
1 answer

Create multiple Google Docs using task queue

I'm trying to create multiples Google Docs in background task. I try to use the taskqueue from Google App Engine but I mustn't understand a point as I kept getting this message : INFO 2016-05-17 15:38:46,393 module.py:787] default: "POST…
Kariamoss
  • 542
  • 1
  • 9
  • 29
0
votes
1 answer

How to use Google Pull Task Queue REST API outside App Engine?

I'm having trouble dealing with the Pull Task Queue REST API. Whenever I try it says "403 - you are not allowed to make this api call". I'm trying this in my computer, which is obviously out of the App and Compute Engine. I have my Service account…
Cristian
  • 63
  • 9
0
votes
3 answers

Specifying retry limit for tasks queued using GAE deferred library

We are offloading certain time consuming tasks using the GAE deferred library and would like to know how can we set the retry limit for those offloaded tasks. We are running into issues where certain tasks are retried for ever as the task would…
keerthy
  • 236
  • 1
  • 9
0
votes
1 answer

How can I make this less cpu intensive?

I have a thread which is running in a loop and executing tasks. outTask::Task* task; while (!m_out_stop) { println("abc"); while (m_outQueue.pop(task)) { println("123"); task->execute(); } } How can I make this less CPU…
KaareZ
  • 615
  • 2
  • 10
  • 22
0
votes
1 answer

Google App Engine Task Queue REST API for the Java Development Server (devserver)

I have a GAE app running on the Java Development Server (AKA devserver) on my local machine that defines some Task Queue pull queues. I want to use the Task Queue REST API from a different standalone Java app, running on my machine, to access these…
yair
  • 8,945
  • 4
  • 31
  • 50
1 2 3
…
50 51