How to understand guava cache stats metrics?

Question

I am using Guava cache in one of my library and that library is being used by some other service (managed by other team) in my company. From my library I am logging cache stats every 50000 times. Below is the the output I see whenever it is getting logged

Cache Stats= CacheStats{hitCount=3296628, missCount=1353372, loadSuccessCount=1353138, loadExceptionCount=0, totalLoadTime=2268064327604, evictionCount=1325410}
Cache Stats= CacheStats{hitCount=3334167, missCount=1365834, loadSuccessCount=1365597, loadExceptionCount=0, totalLoadTime=2287551024797, evictionCount=1337740}
Cache Stats= CacheStats{hitCount=3371463, missCount=1378536, loadSuccessCount=1378296, loadExceptionCount=0, totalLoadTime=2309012047459, evictionCount=1350990}
Cache Stats= CacheStats{hitCount=3407719, missCount=1392280, loadSuccessCount=1392039, loadExceptionCount=0, totalLoadTime=2331355983194, evictionCount=1364535}
Cache Stats= CacheStats{hitCount=3443848, missCount=1406152, loadSuccessCount=1405908, loadExceptionCount=0, totalLoadTime=2354162371299, evictionCount=1378654}

And my cache configuration is as shown below:

    CacheBuilder
      .newBuilder()
      .maximumSize(1000000)
      .expireAfterWrite(120, TimeUnit.SECONDS)
      .concurrencyLevel(5000)
      .removalListener(
          RemovalListeners.asynchronous(new CustomListener(),
              Executors.newSingleThreadScheduledExecutor())).build();

Can anyone help me understand what is the hit ratio we have and how to interpret above cache metrics? I am confuse what does those numbers tells me. Basis on above results can we see better performance if we bump up maximumSize or expireAfterWrite interval?

Note: we are still using guava-11.0.2 and I cannot bump up for some reason.

What don't you understand in the docuentation? http://google.github.io/guava/releases/snapshot/api/docs/com/google/common/cache/CacheStats.html — JB Nizet, Mar 07 '17 at 21:31
You may want to use `refreshAfterWrite` so that the hottest entries are reloaded prior to expiration to minimize blocking the consumers. — Ben Manes, Mar 08 '17 at 00:44

score 5 · Answer 1 · answered Mar 07 '17 at 21:54

From the documentation..

Cache statistics are incremented according to the following rules:

When a cache lookup encounters an existing cache entry hitCount is incremented.
When a cache lookup first encounters a missing cache entry, a new entry is loaded.
1. After successfully loading an entry missCount and loadSuccessCount
  are incremented, and the total loading time, in nanoseconds, is added to totalLoadTime.
2. When an exception is thrown while loading an entry, missCount and loadExceptionCount are incremented, and the total loading time, in nanoseconds, is added to totalLoadTime.
3. Cache lookups that encounter a missing cache entry that is still loading will wait for loading to complete (whether successful or not) and then increment missCount.
When an entry is evicted from the cache, evictionCount is incremented.
No stats are modified when a cache entry is invalidated or manually removed.
No stats are modified by operations invoked on the asMap view of the cache

Now coming to your metrics..

Looking at your metrics..
CacheStats{hitCount=3296628, missCount=1353372, loadSuccessCount=1353138, loadExceptionCount=0, totalLoadTime=2268064327604, evictionCount=1325410} Given that your size is 1M and your ~75% requests are getting a cache hit, and also that your data size is around 1.35M worst case, and your cache misses look like happening due to expiry. I would suggest to increase the maximumSize to roughly 1.25-1.35M and run the same tests again.

It would also help if you mention the timeframe over which the tests are run so that eviction rate can be calculated. If possible, increase the expireAfterWrite time to a higher number to reduce the number of evictions.

Thanks for detailed explanations. Actually it's not a test, one machine is constantly running this build in production from past 5-6 days and we log every 50000 these metrics. How do you know data size is around 1.35M, just curious? So you mean to say I should increase `maximumSize` to `1350000` instead of `1000000`? — , Mar 07 '17 at 22:46
He's guessing that expiration is the cause of evictions, which is likely for a large cache, and that `maximum size / hit-rate` would be the total working set. That may not be true due to "one hit wonders" which pollute LRU caches. Usually trial and error, e.g. using a canary machine with a different configuration, is the simplest solution. Alternatively you can capture an simulate a trace to see the hit rate curve. — Ben Manes, Mar 08 '17 at 00:40

How to understand guava cache stats metrics?

1 Answers1