6

How much data is too much for on-heap cache like ehcache?

I'm getting a 24GB RAM server. I'll probably start off devoting 2-4 GB for caching but may end up devoting 20GB or so to cache. At what point should I worry that GC for on-heap cache will take too long?

By the way, is DirectMemory the only open source off-heap cache available? Is it ready for prime time?

Continuation
  • 12,722
  • 20
  • 82
  • 106
  • The answer to your question depends more on the cache use characteristics than the cache size. For example, read / write ratio, TTL, size of the objects you are storing, the number of objects you could store. Also, your question is asking a pretty fuzzy question. What would you consider "too long?" or "Ready for prime time?" You have to know what kind of SLA you need before you can really evaluate any caching solution. Max response time in ms, and SLA hit percentage. – allingeek Jan 20 '12 at 06:46

3 Answers3

3

Depends on your JVM and especially the used GC. Older GCs especially were not really capable of handling really large heaps, but there's been an increasing effort to fix that.

Azul systems for example sells hardware with hundreds of GB of heap without problems (ie gc pauses in the ms not half minutes) thanks to their special GC, so it's no limitation of Java per se. No idea how good hotspot/IBM have got over time though. But then a 24gb heap isn't that large anyhow - G1 should probably do a good enough job there anyhow.

Voo
  • 29,040
  • 11
  • 82
  • 156
2

At what point should I worry that GC for on-heap cache will take too long?

How long is too long?

Seriously, if the you are running a "throughput" garbage collector and this is giving you pauses that are too long, then you should try switching to a low-pause collector; e.g. CMS or G1.

Stephen C
  • 698,415
  • 94
  • 811
  • 1,216
2

The main problem with a large cache is the full GC time. To give you an idea it might be 1 seconds per GB (This varies from application to application) If you have a 20 GB cache and your application pauses for 20 seconds every so often is that acceptable?

As a fan of direct and memory mapped files I tend to think in terms of when would I not both to put the data off heap, and just use the heap for simplicity. ;) Memory mapped files have next to no impact on the full GC time regardless of size.

One of the advantages of using a memory mapped file is it can be much larger than your physical memory and still perform reasonably well. This leaves the OS to determine which portions should be in memory and what needs to be flushed to disk.

BTW: Having a faster SSD also helps ;) The larger drives also tend to be faster as well. Check for the IOPs they can perform.

In this example, I create an 8 TB file memory mapped on a machine with 16 GB. http://vanillajava.blogspot.com/2011/12/using-memory-mapped-file-for-huge.html

Note, it performs better in the 80 GB file example, 8 TB is likely to be over kill. ;)

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130