Alternative to polling for job scheduler

Question

We've got need for a job server here at work and I'm currently playing with Quartz.net, but the idea of creating my own appeals to me. At the very least understanding what Quartz.net may be doing under the covers can't hurt my understanding/chances of using it more effectively.

So my question is how would you go about getting and firing jobs off on threads with out polling? If you check your 'jobstore' ever 2 min for jobs that need to be fired you'd have a delay of aprox 2min possible. If you decrease the polling time down you increase the strain on your jobstore, and still don't get a true start time. You could preload jobs for the next two min segment and put the threads to sleep for the remaining time so they start at the proper time but that seems kludgy and prone to issues if your poll time is great (deletion,rescheduling,etc) I'm disecting Quartz to figure out how it does it, but I wondered if I was missing something fundamental.

EDIT:

A threaded structure like Kevin initially described seems to be how you SHOULD do a job server. It gives you maximum flexibility with the least amount of overhead. Because threads are such a pita for most people to work with (maybe just me :) the simpler polling example will get the job done in 90% of the cases at the cost of a loss of flexibility and more overhead.

On the flip side unless you are making it single threaded with a single job being executed your going to have to deal with the threading anyway. Might as well go full boar and figure out the signalling.

I also agree with Kevin that the things you are asserting you get for free in a polling db example aren't really free. Your going to code the stuff just like you would if it was threaded/waiting app. What if your polling db job server blows up in the middle of a job? Both would be relying on some durable storage to keep track of their state in case of disaster.

What if you move the 'jobstore' up a level of abstraction and it isn't based on a normal ACID (right term?) database. Now what I believe alot of your 'free' stuff was based on is no longer available (transactions?).

Answer 1

You would create a queue of tasks to be fired off, ordered by time (shortest to longest time). There would be exactly one thread that waits for the time at the head of the queue. When that time expires you remove the item and fire off the task. If it's a repeating task recalculate and put back in queue.

The only tricky thing is the thread should wait based on some condition variable. The condition variable can be signaled if the head of queue changes. Usually with condition variables you can tell if it was signaled vs. a timeout expired. So if it was signaled you just re-wait on the new head time, otherwise the timeout indicates that it's time to run the task at the head of the queue.

This solution means you have only one thread to manage the tasks, and it's not polling.

EDIT:

I'll update my solution to point out that it probably isn't a very good idea to write your own scheduler given that quartz is a very nice solution. Also I think polling every second is kind of overkill, since jobs are typically run on a minute by minute basis, not a per second basis. For example, do you really care if a job starts at 12:00:05, or is 12:00:00 good enough? At any rate you can parameterize your polling to meet the level of granularity that is needed.