optimal java threadsafe object pooling

Question

I am not that familiar with the concurrent library of Java, so for the following problems I would normally just write my own mutex governed code, but I am concerned that with servlet traffic, the mutexes will slow down the system.

The first need is that with a finite set of String keys, I need to lookup first, otherwise create and publish an expensive object. This implies one global mutex on a naive implementation. Is there something better?

The second need is that each expensive object has a soft pool of equivalent workers, any one of which is sufficient for execution. These workers are less expensive to create than the factory for the workers, but they are still expensive and need to be pooled. A naive implementation would have one mutex per factory, and checkout a worker from the soft cache or create it if none available. But with a lot of servlet invocations using the same factory (likely) this mutex also would become a point of contention.

Of course, for the 2 mutexes, I can absolutely minimize the time spent in the synchronized statement, but I'm looking for something better in both cases. Maybe there's a nonblocking solution for both?

Andy

Answer 1

For the first part: Instead of putting the expensive objects directly into the HashMap, instead create a simple wrapper that is cheap to create. You then basically create the expensive object on demand in the wrappers getExpensiveObject() method - although it's obviously possible to trigger the creation instantly if that's preferred. In either case you have to synchronize the get method, but that can be done cheaply with double checked locking - in general we just replace a normal read with a volatile read and have the expensive synchronization only while creating the object.

This assumes you're using a ConcurrentHashMap of kinds, since we need putIfAbsent or some equivalent method for this to work (we don't want to replace an existing expensive object with our empty wrapper)

No time to think about the second problem at the moment, maybe later.

Answer 2

Take a look at this tutorial on thread pools. It appears to describe what you're looking for, a pool of worker threads.

Answer 3

I need to lookup first, otherwise create and publish an expensive object.

There is no nonblocking solution to this. There is a nonblocking map with get and put if absent, but that requires the put value to be precomputed which you can't do with an expensive object.

You might want to take a look at "Generic and concurrent object pools" which uses some linked blocking queue tricks that might avoid single mutex contention.

Answer 4

I think I have the answer after futher thought:

NOTE: my answer depends on the fact that both the factory is idempotent relative to its String key and the workers are all idempotent relative to their factory which may not have been obvious from the question.

For the first global singleton key based hashmap, I leverage the idea that the hashmap never has deletions. Only new idempotent ones. So I use a volatile reference to the hashmap, and get the current map from the volatile singleton variable without a mutex. (volatile gets of references are now very contention cheap in Java) If the map is stale, in the sense that it doesn't have all of the current factories, that's ok. Because if it does have the factory (it usually will once warmed up), I've got it. With only the cost of the volatile get. If it doesn't have it, I now consult the live map with a get inside the mutex for the "live" map. If I get the factory from the map now (unlikely) I've got it. Otherwise, I now do the very expensive operation of creating the factory (outside mutexes). When done, I got back to the live map with a mutex, and due to another thread doing the same thing, it may be there now! So if the factory is there in the map, I throw away the wasted factory that I just created and use the one that was put there ahead of me. Otherwise, I stuff the new factory in the map, leave the mutex, and start using the factory.

I don't think there's better for this.

On the factory softcache, I think I just want to leverage ConcurrentLinkedQueue. However, my elements will be references to softreferences. So I checkout from the ConcurrentLinkedQueue an object that has a reference to a soft reference to the worker itself. The worker may have been softreleased, so I just create the worker from the factory again and recreate the softreference in the object that I got from the ConcurrentLinkedQueue, and set a softreference to the worker. So there's no mutex for getting a worker from the factory, just the ConcurrentLinkedQueue checkout, and the references to the worker are soft.