Caffeine: Use stale values when AsyncLoader fails to refresh

Question

I want to configure my Caffeine cache to return stale results when loader fails to refresh the cache. The following Kotlin code demonstrates the case:

    @Test
    fun `completeble future`() = runBlocking {
        val cache = Caffeine.newBuilder()
            .refreshAfterWrite(Duration.ofSeconds(1))
            .expireAfterWrite(Duration.ofSeconds(1))
            .buildAsync<String, String> { key: String, executor ->
                GlobalScope.future(executor.asCoroutineDispatcher()) {
                    throw Exception("== Error ==")
                }
            }

        cache.put("id", CompletableFuture.completedFuture("value"))

        delay(2000)

        assertEquals("value", cache.get("id").await())
    }

I expect this test to pass but instead I get the following error:

WARNING: Exception thrown during asynchronous load
java.lang.Exception: == Error ==
    at fsra.manager.TranslationManagerImplTest$completeble future$1$cache$1$1.invokeSuspend(TranslationManagerImplTest.kt:93)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)




java.lang.Exception: == Error ==

    at fsra.manager.TranslationManagerImplTest$completeble future$1$cache$1$1.invokeSuspend(TranslationManagerImplTest.kt:93)
    at |b|b|b(Coroutine boundary.|b(|b)
    at fsra.manager.TranslationManagerImplTest$completeble future$1.invokeSuspend(TranslationManagerImplTest.kt:101)
Caused by: java.lang.Exception: == Error ==
    at fsra.manager.TranslationManagerImplTest$completeble future$1$cache$1$1.invokeSuspend(TranslationManagerImplTest.kt:93)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)

I wrote the code in Kotlin but I don't think the issue is related to Kotlin coroutines. I want to configure Caffeine to not throw when refreshing, but instead return the previous result in the cache.

do you see another `caused by` exception in the stack trace? — sidgate, May 22 '20 at 17:44
Yes @sidgate. I have added the whole exception details to the question. — alisabzevari, May 23 '20 at 12:03

Ben Manes · Accepted Answer · 2020-05-22T19:24:45.363

6

The cache is set to expire entries after 1 second and the test waits 2 seconds. The next call then forces it to be loaded anew because the entry is unusable, and you throw an exception.

The refresh has no effect when it is greater-or-equal to the expiration time. When less than, the entry is stale but usable so it is returned and asynchronously reloaded. This is to allow popular items, like a configuration, to stay in the cache without a periodic penalty on a reload. The unpopular items are those not accessed within the expiration interval and are allowed to be evicted. If the refresh was unable to succeed then expiration would kick in and the entry would be dropped, as the expiration sets the maximum time it is considered usable.

A greater expiration value, like 5 seconds, would pass your test. If your use-case is instead to blindly reload all of the cache's contents periodically, you can instead use a regular Map and a ScheduledExecutorService to refresh it.

edited May 22 '20 at 19:24

answered May 22 '20 at 19:16

Ben Manes

9,178
3
35
39

Thanks for the answer. What I want is to fallback to the current cached value even if it is stale. It looks like Caffeine is overkill (or not appropriate) for this usecase, right? – alisabzevari May 23 '20 at 12:23
@alisabzevari That could work except that the expiration time forces it to be evicted, so you would need to set it higher. However it isn't a fallback, it returns the stale value and triggers a refresh. If you want instead to perform a blocking reload and then fallback if a failure, you would need to build that logic yourself. That could leverage caffeine, e.g. a victim cache fallback layer, but it is too custom per usage to offer a built-in pattern. We could discuss over a github issue / email, as it is a larger topic to explore. – Ben Manes May 23 '20 at 18:35
That makes sense. I meant fallback to previous value. – alisabzevari May 24 '20 at 12:22

Caffeine: Use stale values when AsyncLoader fails to refresh

1 Answers1