Apache Pool 2 library - connections not closed automatically

Question

I have the following code in Scala that uses the Apache pool2 library. The object to pool is OlapConnection (an olap4j class, similar to an SQL connection).

Problem is that I can't make the pool close the connections automatically when the number of pooled objects exceeds the maximum.

If I return an object to the pool (with pool.returnObject) that triggers passivateObject. If I close the connection in passivateObject, I would close it every time I return an object, and that's not what I want - I need the open connections to be cached. If I don't close the connection in passivateObject, then it will never be closed.

How to make this work?

Scala code:

class OlapConnectionUtil (val pool: ObjectPool[OlapConnection]) {
     def connect = {
        pool.borrowObject
     }
     def close(olapConnection: OlapConnection) = {
       pool.returnObject(olapConnection)
     }
}

class OlapConnectionFactory extends BasePooledObjectFactory[OlapConnection] {

  override def create = {           
      val connectionString = "jdbc:mondrian:Jdbc=jdbc:mysql://localhost:3306/foodmart?" + 
            "user=x&password=x;Catalog=FoodMart.xml;JdbcDrivers=com.mysql.jdbc.Driver"           
       val connection = DriverManager.getConnection(connectionString)
       connection.unwrap(classOf[OlapConnection])
    }

    override def wrap(olapConnection: OlapConnection) = 
                                  new DefaultPooledObject(olapConnection)

    override def passivateObject(pooledObject: PooledObject[OlapConnection] ) {
      println("passivateObject WAS CALLED")
      pooledObject.getObject.close
    }    

  }


class Test  {

  val olapConnectionFactory = new OlapConnectionFactory

  def test = { 

     val config = new GenericObjectPoolConfig
     config.setMaxIdle(5)
     config.setMaxTotal(10)

     val util = new OlapConnectionUtil(
                 new GenericObjectPool[OlapConnection](olapConnectionFactory,config))
     val olapConnection = util.connect

     // do stuff with olapConnection

     util.close(olapConnection)

score 2 · Accepted Answer · answered May 02 '18 at 20:32

If you open the JavaDocs for PooledObjectFactory you may see that

passivateObject(org.apache.commons.pool2.PooledObject<T>) is invoked on every instance when it is returned to the pool.

destroyObject(org.apache.commons.pool2.PooledObject<T>) is invoked on every instance when it is being "dropped" from the pool (whether due to the response from validateObject(org.apache.commons.pool2.PooledObject<T>), or for reasons specific to the pool implementation.) There is no guarantee that the instance being destroyed will be considered active, passive or in a generally consistent state.

In other words you should put your resource de-allocation logic into the destroyObject. Here is a modified test of yours with a fake OlapConnection implementation

import org.apache.commons.pool2._
import org.apache.commons.pool2.impl._

// fake!
case class OlapConnection(val id: Int) {


  def close(): Unit = {
    println(s"Close was called for $this")
  }
}

class OlapConnectionUtil(val pool: ObjectPool[OlapConnection]) {
  def connect = {
    pool.borrowObject
  }

  def close(olapConnection: OlapConnection) = {
    pool.returnObject(olapConnection)
  }
}

class OlapConnectionFactory extends BasePooledObjectFactory[OlapConnection] {
  var cnt = 0

  override def create = {
    cnt += 1
    new OlapConnection(cnt)
  }

  override def wrap(olapConnection: OlapConnection) =
    new DefaultPooledObject(olapConnection)

  override def passivateObject(pooledObject: PooledObject[OlapConnection]) {
    println("-passivateObject was called")
  }

  override def destroyObject(pooledObject: PooledObject[OlapConnection]): Unit = {
    super.destroyObject(pooledObject)
    println("--destroyObject was called")
    pooledObject.getObject.close
  }
}

object Test {

  val olapConnectionFactory = new OlapConnectionFactory

  def test = {

    val config = new GenericObjectPoolConfig
    config.setMaxIdle(5)
    config.setMaxTotal(10)

    val util = new OlapConnectionUtil(new GenericObjectPool[OlapConnection](olapConnectionFactory, config))
    val initConnections = (1 to 10).map(i => util.connect).toList
    val closeCons = initConnections
    Thread.sleep(100)
    println("Start closing")
    closeCons.zipWithIndex.foreach(ci => {
      Thread.sleep(100)
      println(s"Before close ${ci._2 + 1}")
      util.close(ci._1)
      println(s"After close ${ci._2 + 1}")
    })
  }
}

The output produced by this code is:

Before close 1
-passivateObject was called
After close 1
Before close 2
-passivateObject was called
After close 2
Before close 3
-passivateObject was called
After close 3
Before close 4
-passivateObject was called
After close 4
Before close 5
-passivateObject was called
After close 5
Before close 6
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(6)
After close 6
Before close 7
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(7)
After close 7
Before close 8
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(8)
After close 8
Before close 9
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(9)
After close 9
Before close 10
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(10)
After close 10

As you can see in this output close is being called since OlapConnection(6) as one would expect given the pool configuration.

In the `create` method of `OlapConnectionFactory` you are instantiating a new object with `new OlapConnection`, instead I need to retrieve an existing connection from the pool, how do I do that? — ps0604, May 03 '18 at 00:46
@peterSc, You might see that in my example I use a **_fake_** class instead of the real `OlapConnectionFactory` because I don't want to set up all the real infrastructure which is unrelated to how to use the pool properly. — SergGr, May 03 '18 at 13:57

Apache Pool 2 library - connections not closed automatically

1 Answers1