Cache with long running computations of values

Question

I need to store objects in a cache and hese objects take a long time to create. I started with ConcurrentHashMap<id, Future<Object>> and everything was fine, until Out of Memory started to happen. Moved to SoftReferences and it was better, but now I need to control eviction. I'm in the process of moving to Ehcache.

I'm sure there is a library for such thing but I really need to understand the logic of doing the cache storage and calculation in two phases, while keeping everything consistent and not recalculating something that is already calculated or in the process of being calculated. Is a two level cache, one for the more persistent result and the other for the in the process of being calculated.

Any hints on how to better the following code which I'm sure has concurrency problems in the Callable.call() method?

public class TwoLevelCache {

    // cache that serializes everything except Futures
    private Cache complexicos = new Cache();

    private ConcurrentMap<Integer, Future<Complexixo>> calculations = 
        new ConcurrentHashMap<Integer, Future<Complexico>>();

    public Complexico get(final Integer id) {

        // if in cache return it
        Complexico c = complexicos.get(id);
        if (c != null) { return c; }

        // if in calculation wait for future
        Future f = calculations.get(id);
        if (f != null) { return f.get(); } // exceptions obviated

        // if not, setup calculation
        Callable<Complexico> callable = new Callable<Complexico>() {
            public Complexico call() throws Exception {
                Complexico complexico = compute(id);
                // this might be a problem here
                // but how to synchronize without
                // blocking the whole structure?
                complexicos.put(id, complexico);
                calculations.remove(id);
                return complexico;
            }
        };

        // store calculation 
        FutureTask<Complexico> task = new FutureTask<Complexico>(callable);
        Future<Complexico> future = futures.putIfAbsent(id, task);
        if (future == null) {
            // not previosly being run, so start calculation
            task.run();
            return task.get(); // exceptions obviated
        } else {
            // there was a previous calculation, so use that
            return future.get(); // exceptions obviated
        }

    }

    private Complexico compute(final Integer id) { 
        // very long computation of complexico
    }

}

Answer 1

And what do you do with the values once they are calculated? What is the number of new calculations per second?

If they are used (stored) and then disposed then I think that Reactive approach (RxJava and similar) could be a nice solution. You could put your "tasks" (a POJO with all info needed to perform calculation) on some off-heap structure (it could be some persistent queue etc.) and only perform calculations for as many as you want (throttle the process with the number for computational threads you want to have).

This way you would avoid OOM and would also gain much more control over the entire process.