Probabilistic Early Expiration with Guava Cache

Question

I have a Guava cache created with expireAfterWrite(20, TimeUnit.MINUTES). The cache is also initialized with dozens of entries when created. When values expire, client is supposed to call an external service and refresh the cache.

Problem is - all the initialized values would expire simultaneously and after 20 minutes, and the client would call the service dozens of times almost at the same instance.

I don't want this to happen. To avoid this - one idea is to probabilistically expire the entries a little earlier than the TTL so that the service won't get hit a lot at the same time.

Unfortunately, I don't see an option of doing this with Guava cache - at least nothing popped up from the Wiki or Javadoc. Is there any other library available? I am trying to avoid writing my own implementation of a cache for this purpose.

Guava uses an O(1) expiration order, so this is not directly possible. This is somewhat mitigated by also using `refreshAfterWrite()` so that actively used entries are reloaded without blocking other readers. Otherwise its left to your own code to batch, re-warm, use a secondary layer, etc. to construct more a powerful policy. — Ben Manes, Oct 08 '16 at 03:56

dimo414 · Answer 1 · 2017-12-06T08:57:15.867

Cache warming is a fairly involved topic, so there are a lot of reasonable approaches but not necessarily a single "correct" one. An easy option would be to simply trigger additional writes at staggered times in the future in order to reset the expiry times:

public static <K, V> void staggerCacheExpiration(
    Cache<K, V> cache, long maxExpiration, TimeUnit unit, ScheduledExecutorService scheduler) {
  for (Entry<K, V> e : cache.asMap().entrySet()) {
    long delay = ThreadLocalRandom.current().nextLong(0, maxExpiration);
    scheduler.schedule(() -> cache.put(e.getKey(), e.getValue()), delay, unit);
  }
}

A critical problem with any approach that relies on Cache itself is you'll end up trigger a (potentially expensive) refresh only when a caller needs the value. It may be better not to use expireAfterWrite() or refreshAfterWrite() and instead run a dedicated thread that's responsible for updating each key in sequence. By having a single thread updating all keys you'll naturally avoid any hotspots where multiple keys are being refreshed simultaneously, and also avoid blocking any threads relying on values from the cache.

As Ben Manes suggests you might prefer to encapsulate your Cache in your own type so whatever behavior you choose isn't exposed to your callers.

Probabilistic Early Expiration with Guava Cache

1 Answers1