Questions tagged [job-queue]

In system software, a job queue (sometimes batch queue), is a data structure maintained by job scheduler software containing jobs to run.

A job queue is a queue (that is, a First In First Out (FIFO) collection) of tasks to be carried out at an unspecified time in the future. Workers may take on the task at the head of the queue and carry it out, returning to take another when they are finished.

This is often applied to multi-threaded programming environments, in which there are fewer threads than available tasks so tasks must be queued until a thread is available to process them.

113 questions
1
vote
3 answers

Java Callback Execution in Thread Executed After run()

I'm using a job queue library, where you define Jobs and post them to a JobManager. The JobManager has a ThreadGroup with workers that pull Jobs off of a queue and call Job.onRun(). The amount of threads are set by giving a max and min amount of…
Eliezer
  • 7,209
  • 12
  • 56
  • 103
1
vote
0 answers

Properly implementing job queues using gearman, getting status from subtasks

What I want to achieve: User uploads a datafile. It is added to queue, and when it's turn comes, it creates number of parallel tasks consisting of converting rows inside that file. Anytime user wants, he can get status (in %), of how much rows were…
Misiur
  • 5,019
  • 8
  • 38
  • 54
1
vote
0 answers

managing ironworker task queue delays

I have a project where we make a request from an iPhone client to our API, the api spawns an ironworker task, if the task completes it notifies the iPhone client and several Android phones that all went well and they should all connect to a socket.…
jdog
  • 10,351
  • 29
  • 90
  • 165
1
vote
2 answers

MSMQ as a job queue

I am trying to implement job queue with MSMQ to save up some time on me implementing it in SQL. After reading around I realized MSMQ might not offer what I am after. Could you please advice me if my plan is realistic using MSMQ or recommend an…
Vojtiik
  • 2,558
  • 1
  • 17
  • 21
1
vote
1 answer

Zend Framework ZEND_QUEUE implementation

I am new to ZEND and I am working on the version (ZEND 1.11.1) I am trying to implement the ZEND_QUEUE in my zend application and there is no proper tutorial for me. All through I am trying to implement the Queue. I am developing a Queue for DB. The…
DonOfDen
  • 3,968
  • 11
  • 62
  • 112
1
vote
0 answers

How a job queueing software works

I have a small website where and i am allowing people to sign up for accounts.I want after someone has signed up,he/she is sent a welcome email and a confirmation email.I want to attempt to do this using a job queuing system. I have never used a…
Gandalf
  • 1
  • 29
  • 94
  • 165
1
vote
0 answers

Http xmpp multithread process

I am using a python library called sleekxmpp to imeplement xmpp protocol. Actually, the whole process is, a client send a http request to me, then I need to send xmpp messages to other clients through my sleekxmpp library. I need to gather those…
1
vote
2 answers

Is it good to have job details in the job queue table itself or have the details in a separate table?

I am currently designing a module where a service or console application will be reading Job from a JobQueue table. I have designed the JobQueue table with the following info in it: JobId: int Primarykey JobType: int notnull State: varchar…
Vinoth
  • 2,419
  • 2
  • 19
  • 34
0
votes
1 answer

Have any good links/articles on job queuing & db sharding?

Does anyone have good links on how/when/why to use job queuing to scale web apps? Also, articles on db sharding would be useful too :)
Justin Vincent
  • 1,329
  • 2
  • 14
  • 22
0
votes
1 answer

How to retrieve the list of scheduled jobs in a job queue?

I am assigned with a task to check if a job is currently present in a job queue as scheduled job. The matching criteria is the name of the job - as long as the job name matches the specified one then it counts. So is there any simple way to return…
God_of_Thunder
  • 753
  • 3
  • 20
  • 46
0
votes
1 answer

Can I implement a redis job queue with plain bash and redis-cli?

I am relatively new to redis, and I am looking to implement a job queue as a simple list of job ids (and nothing else in redis). I've searched around, but couldn't find examples of how to do this in plain shell interface (redis-cli). The examples…
thor
  • 21,418
  • 31
  • 87
  • 173
0
votes
1 answer

How to change job parameters in telegram app using telegram.ext functionality?

I'm trying to understand how jobs working in telegram.ext module and writing small bot for this: user should be able to set time for reminder and change this time. I created a job using job_queue.run_daily: context.job_queue.run_daily(alarm,…
Denis
  • 3
  • 3
0
votes
1 answer

Node queuing jobs

I have a MethodQueue class which enqueue method in queue array and execute them. The tricky part here is that I want to execute one job at a time and next job has to start after some x seconds after previous completed to avoid exceeding request…
0
votes
0 answers

Add fields to Laravel job records?

I have a working Laravel 9 queued job class using the 'database' driver. I would like to show these jobs on a website and therefore I would like to add and save a value on a custom field on these jobs records when they are constructed.
0
votes
1 answer

Remove duplicated outputs from BullJS job queue

I am running a nodejs/express API server on Heroku (my web server dyno) that queues up jobs in Redis that are managed by BullJS on a worker server (a separate worker dyno) These jobs can take a while (up to 2 hours). However, I have noticed that…
user2497586