I have configured two Java WARs with quartz schedulers (version 2.2.1) starting with XMLSchedulingProcessorPlugin. Both web applications are also running in a cluster mode (they are deployed in two identical machines), so I enabled the properties for quartz:
#===========================================================================
# Clustering
#===========================================================================
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 60000
Both applications are running in JBOSS AS 7.1 configured with Quartz´s JobStoreTX. They save their jobs, triggers and so on into a MySQL database, which currently is configured with Galera DB (1 virtual IP address, 2 real nodes).
Currently, I am testing the failure of one of the real nodes so the jobs keep up firing even when a power outage occurs. In that case I noticed some failures, such as the one described in this Terracotta issue (the patch is not applied in the current version of Quartz).
In my case, I should have 4 Quartz instances in the QRTZ_SCHEDULER_STATE table...even if one of the MySQL nodes restart. The fact is that sometimes one or two instances are deleted from the table (maybe the ones which do not have any active job) so I am afraid it is possible to lose both instances of an application during the cluster recovery.
Has anyone experienced the same? Any other solution than restarting the JBOSS in order to reload Jobs and Triggers?
Thanks in advance.
