0

I have a @Scheduled method that is called each 5 minutes.

The batch does a lot of calculation, it takes about 2-15 minutes to complete (there is a flag that prevents a batch to run if one is already running).

I use lot of Maps to cache the results of the calculation. I could resume it like that. One major Map that will contain the cumulative results, and some Map/List into sub method for internal calculus.

The batch could take around 3-6 Gigs to run.. lot of data to analyse.

I think the Map/List that I used inside a method will be eligible to GC when the code exits the method right?

Should I use a WeakHashMap or WeakReference in my methods or call Map.clear(); before exiting the method?

I want to reduce the memory usage if possible that don't wait at the end of the batch for GC to do a cleanup.

When the batch is complete, the memory will reduce to 500megs-1G max.

Holger
  • 285,553
  • 42
  • 434
  • 765
Sebastien Dionne
  • 757
  • 2
  • 19
  • 34
  • I suppose you don’t want to lose cached values while your method is still running, so weak references/maps are not really useful. Further, it would cause the opposite, having parts of the data collected while you method runs will raise the CPU usage. Freeing the whole data at once at the end imposes no cost, the garbage collector’s resource consumption is proportional to the remaining objects, not to the dead ones. – Holger Feb 08 '16 at 18:14

2 Answers2

0

Should I use a WeakMap for WeakReference in my methods?

Weak reference is good only if you keep other references to your data, and you don't want to prevent that data from being garbage collected. It does not look like this is something that you are doing, because you need that data to continue processing.

or call Map.clear(); before existing the method?

If the map is local to your method, calling clear() is not going to speed up GC in any meaningful way. If, however, your map is a field on an object that remains available after your method exits, then calling clear is an important way of avoiding "lingering memory leaks".

As far as reducing memory footprint goes, the trick is determining the subset of data that you no longer need, and releasing it early. If you need all data all the time, or if you have no way to determine what data you are going to need next, you wouldn't be able to reduce the temporary footprint of your program.

Sergey Kalinichenko
  • 714,442
  • 84
  • 1,110
  • 1,523
0

I get a feeling that you would like to use WeakHashMap. WeakHashMap uses weak references for holding map keys, which allows the key objects to be garbage collected when they are no longer used by the application, and the get() implementation can tell a live mapping from a dead one by whether WeakReference.get() returns null. But this is only half of what is needed to keep a Map's memory consumption from increasing throughout the lifetime of the application; something must also be done to prune the dead entries from the Map after the key object has been collected. Otherwise, the Map would simply fill up with entries corresponding to dead keys. And while this would be invisible to the application, it could still cause the application to run out of memory because the Map.Entry and value objects would not be collected, even if the key is.

Dead mappings could be eliminated by periodically scanning the Map, calling get() on each weak reference, and removing the mapping if get() returns null. But this would be inefficient if the Map has many live entries. It would be nice if there was a way to be notified when the referent of a weak reference is garbage collected, and that is what reference queues are for.

Please refer to this page on Weak Hash Reference

The Roy
  • 2,178
  • 1
  • 17
  • 33