Java job scheduler based on Redis?

Question

I am looking to replace Quartz as a job scheduler in our project. We already use Redis with cluster support as a distributed cache layer and we thought that maybe we could use Redis for job scheduling too. Has anyone implemented job scheduling in Java using Redis? I searched but could not find a library for this purpose. So I am starting to think that this is maybe not a popular solution?

Answer 1

Take a look at Redisson. It allows to schedule and execute tasks (with cron-expression support) using simple ScheduledExecutorService api and based on Redis queue.

Here is an example. First define a task using java.lang.Runnable interface. Each task can access to Redis instance via injected RedissonClient object.

public class RunnableTask implements Runnable {

    @RInject
    private RedissonClient redissonClient;

    @Override
    public void run() throws Exception {
        RMap<String, Integer> map = redissonClient.getMap("myMap");
        Long result = 0;
        for (Integer value : map.values()) {
            result += value;
        }
        redissonClient.getTopic("myMapTopic").publish(result);
    }

}

Submit it to Redis based ExecutorService.

RScheduledExecutorService executorService = redisson.getExecutorService("myExecutor");
RScheduledFuture<?> future = executorService.schedule(new CallableTask(), 10, 20, TimeUnit.MINUTES);

future.get();
// or cancel it
future.cancel(true);

// cancel by taskId
executorService.cancelTask(future.getTaskId());

Examples with cron expressions:

executorService.schedule(new RunnableTask(), CronSchedule.of("10 0/5 * * * ?"));

executorService.schedule(new RunnableTask(), CronSchedule.dailyAtHourAndMinute(10, 5));

executorService.schedule(new RunnableTask(), CronSchedule.weeklyOnDayAndHourAndMinute(12, 4, Calendar.MONDAY, Calendar.FRIDAY));

All tasks are distributed across Redisson nodes. You can run these nodes as much as you need.

Answer 2

How about Redis Labs' redis-quartz:

RedisJobStore A Quartz Scheduler JobStore that uses Redis for persistent storage.

We'd appreciate any feedback you have :)

Answer 3

I used Spring Task Scheduler with Shedlock and Redis to make scheduled task execution distributable.

In my XML configuration file I specify the task scheduler:

<task:scheduler id="myScheduler" pool-size="4"/>

<task:annotation-driven scheduler="myScheduler" />

In SchedulerLockConfiguration.java I create the LockProvider Spring bean and set up the Redis connection and the Jedis connection pool:

@Configuration
@EnableSchedulerLock(defaultLockAtMostFor = "PT10H")
public class SchedulerLockConfiguration {

  @Bean
  public JedisPool jedisPool() {
    JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
    jedisPoolConfig.setMaxTotal(100); // The maximum number of connections that are supported by the pool
    jedisPoolConfig.setMaxIdle(100);  // The maximum number of idle connections in the pool
    jedisPoolConfig.setMinIdle(10);  // The minimum number of idle connections in the pool
    return new JedisPool(
        jedisPoolConfig,
        Constants.REDIS_HOSTNAME,
        Integer.parseInt(Constants.REDIS_PORT),
        Integer.parseInt(Constants.REDIS_SESSION_TIMEOUT),
        null,
        Constants.REDIS_DATABASE_SHEDLOCK);
  }

  @Bean
  public LockProvider lockProvider(JedisPool jedisPool) {
    return new JedisLockProvider(jedisPool, "yourRedisNamespace");
  }

}

...then finally, I use @Scheduled and @SchedulerLock annotations on the methods I want to schedule:

@Scheduled(cron = "${my.cron.expression}")
@SchedulerLock(name = "myScheduler", lockAtLeastFor = "PT10M")
public void processDelayedAutomationRules() { ... }

When a @Scheduled job's cron kicks in, whichever applciation instance / server node puts the lock faster into the Redis store, that'll be the one executing it.