I'm running Quartz.NET (latest) in-process in an ASP.NET app backed by a SQL Server database. I have an administrative process which runs and outputs details of currently running jobs to a log file, using code from Get all jobs in Quartz.NET 2.0.
Here's a sample (and real) output:
-----STARTING JOBS STATUS CHECK at 11/28/2015 5:51:06 PM -----
Job group: UPDATE_TRANSFER_FEED_JOB_GROUP
Job name: 155143_UPDATE_TRANSFER_FEED_JOB
Job data - Key: SourceFeedUpdateID, Value: 155143
Description:
Trigger name: 155143_UPDATE_TRANSFER_FEED_JOB
Trigger group: UPDATE_TRANSFER_FEED_JOB_GROUP
Trigger type: SimpleTriggerImpl
Trigger state: Complete
Previous fire time: 11/28/2015 5:41:00 PM
-----FINISHED JOBS STATUS CHECK-----
Periodically I see certain jobs that show "Trigger state: Complete" but never leave the queue (this might be incorrect terminology). In other words they keep showing up in the log (like shown above) each time the job statuses are outputted, and they stay there indefinitely. Furthermore I have a job listener set for jobs of this type and I see that for the stuck jobs the Listener.JobWasExecuted() is never called. Other jobs complete and disappear from the output as I would expect.
The only way to get these stuck jobs processed is to restart the web app. Immediately after restarting, the listeners' JobWasExecuted() fires, they show 'Job Requests Recovery = Yes' and are gone when the job status logging next runs.
I have a feeling part of this is that I'm not understanding job lifetimes and statuses.
Can you help me understand this?