Caffeine eviction by size seems to not work

Question

I am using caffeine cache.

I want to put it under size limitation but it does not work properly.

test 1:

Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(3)
                .build();

        Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(3)
                .build();

        for (int i = 1; i <= 10; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);
        }

        System.out.println("cache size: " + cache.estimatedSize() + ", cache keys: " + cache.asMap().values().stream().collect(Collectors.joining(",")));


result:   cache size: 10, cache keys: 1,2,10

another test: trying to get key and set max to 1

Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(1)
                .build();

        for (int i = 1; i <= 10; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);

            if (i % 2 == 0) {
                cache.getIfPresent("5");
            }
        }

        System.out.println("cache size: " + cache.estimatedSize() + ", cache keys: " + cache.asMap().values().stream().collect(Collectors.joining(",")));


cache size: 10, cache keys: 2,3,4,5,6,7,8,9,10

last test : run 100 times, max size 1

 Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(1)
                .build();

        for (int i = 1; i <= 100; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);

            if (i % 2 == 0) {
                cache.getIfPresent("5");
            }
        }

        System.out.println("cache size: " + cache.estimatedSize() + ", cache keys: " + cache.asMap().values().stream().collect(Collectors.joining(",")));

cache size: 99, cache keys: 96,97,99,19,23,58

can someone please help me understand this and how to make it work properly?

Thanks to Ben Manes, I added .executor(Runnable::run)

Now after doing this I do get only 3 items

 Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(3)
                .executor(Runnable::run)
                .build();

        for (int i = 1; i <= 10; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);

            if (i % 2 == 0) {
                cache.getIfPresent("5");
            }
        }
        cache.cleanUp();
        System.out.println("cache size: " + cache.estimatedSize() + ", cache: " + CodecUtils.toJson(cache.asMap().values()));


cache size: 3, cache: ["3","9","10"]

wouldn't this block my thread?
why isn't key 5 in the cache as I have been using it several times?

Try disabling async eviction, `Caffeine.executor(Runnable::run)`. That’s causing your test to outpace the eviction and cause surprising results. — Ben Manes, Apr 23 '20 at 06:38

score 3 · Accepted Answer · answered Apr 23 '20 at 16:11

By default the cache will perform some operations asynchronously, such as eviction and notifying a removal listener. This is to minimize request latencies, as the auxiliary work is not necessary for the request itself and the user supplied callbacks might be expensive.

The cache's own maintenance work is very cheap, so you can safely run it on the caller's thread if desired by using Caffeine.executor(Runnable::run). This will penalize the caller with additionally evicting the entry, but will not block other operations from occurring. This is due to the cache internally using multiple locks and operation buffers, so that it can schedule work when a lock is busy rather than block threads.

In regards to the size, this is because the entry is evicted prior to being retrieved so it doesn't build up frequency. A getIfPresent doesn't increase the frequency if the entry is absent, whereas get(key, /* loading function */) would because it is penalized to load the value on the miss. The eviction policy utilizes both recency and frequency in its decisions, so it may evict recent arrivals early as often "one-hit wonders", aka cache pollution.

If we take your code as is and output the cache's state we see this,

for (int i = 1; i <= 10; i++) {
  String val = String.valueOf(i);
  cache.put(val, val);
  System.out.println(val + " -> " + cache.asMap());
  if (i % 2 == 0) {
    cache.getIfPresent("5");
  }
}
cache.cleanUp();
System.out.println("cache size: " + cache.estimatedSize());

1 -> {1=1}
2 -> {1=1, 2=2}
3 -> {1=1, 2=2, 3=3}
4 -> {2=2, 3=3, 4=4}
5 -> {2=2, 3=3, 5=5}
6 -> {2=2, 3=3, 6=6}
7 -> {2=2, 3=3, 7=7}
8 -> {2=2, 3=3, 8=8}
9 -> {2=2, 3=3, 9=9}
10 -> {2=2, 3=3, 10=10}
cache size: 3

If we access key 5 on every iteration then it is retained,

for (int i = 1; i <= 10; i++) {
  String val = String.valueOf(i);
  cache.put(val, val);
  System.out.println(val + " -> " + cache.asMap());
  cache.getIfPresent("5");
}
cache.cleanUp();
System.out.println("cache size: " + cache.estimatedSize());

1 -> {1=1}
2 -> {1=1, 2=2}
3 -> {1=1, 2=2, 3=3}
4 -> {2=2, 3=3, 4=4}
5 -> {2=2, 3=3, 5=5}
6 -> {3=3, 5=5, 6=6}
7 -> {3=3, 5=5, 7=7}
8 -> {3=3, 5=5, 8=8}
9 -> {3=3, 5=5, 9=9}
10 -> {3=3, 5=5, 10=10}
cache size: 3

Thanks for the ansewr. I want to add Weigher so the cache would not cause the ram explode. Would it hurt performance? — Ido Barash, Apr 26 '20 at 05:32
I suppose only if the weigher itself is very slow, but not from the cache's own logic. — Ben Manes, Apr 26 '20 at 05:34
What is the best way to put a Weigehr? I tried using `com.carrotsearch.sizeof.RamUsageEstimator`. I need size in bytes or something like that. — Ido Barash, Apr 26 '20 at 05:36
I don't know. [MemoryMeter](https://github.com/jbellis/jamm) was a similar library with a few estimation strategies. — Ben Manes, Apr 26 '20 at 05:45

Caffeine eviction by size seems to not work

1 Answers1