4

Locally when I am developing my application I launch my play2 application using sbt run

I love how I can make code changes, and then reload my browser to see my changes.

After about roughly 10 code changes or so, I get a postgresql too many connection error (see below).

My db connection is using the below DatabaseAccess.scala class.

I'm guessing on each reload it is creating a bunch of connections to postgresql. In my Global am currently doing:

override def onStart(app: Application) {
    Logger.info("Global.onStart")

    DatabaseAccess.loadConfiguration()
  }

In production I am also worried that if I make multiple deploys in a short period of time, or start/stop my service I might run into this problem also.

How can I ensure all connections are destroyed? I know I can either put something in the onStart or OnStop but I'm not really sure how to clear the connections that might have existed previously. (assuming this isn't a bug)

I did create a release() method that I was calling on onStop but that didn't seem to work:

 def release() {
    configs = Map[String, BoneCPConfig]()
    dataSources = Map[String, BoneCPDataSource]()
    databases = dataSources.map { case(key, value) => (key, Database.forDataSource(value)) }
  }

All I did was re-init the vars so I guess that wasn't really what I needed to do.

My data access pattern is as follows:

def getById(userId: Int): Option[User] = {
    db.withSession { implicit session =>
      return userDao.getById(userId)
    }
  }

My database access class looks like:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/unfiltered/src/main/scala/DatabaseAccess.scala

My application.conf connection looks like:

#postgresql
db.default.driver="org.postgresql.Driver"
db.default.url = "jdbc:postgresql://localhost/testweb_development"
db.default.user = "testdbuser"
db.default.password = ""
db.default.minConnections = 4
db.default.maxConnections = 24
db.default.maxThreads = 2

Locally when developing after about approximately 10 code changes and sbt doing it's nice partial class reloads I get this error:

play.api.Application$$anon$1: Execution exception[[SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:postgresql://localhost/testweb_development, username = testdbuser. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
org.postgresql.util.PSQLException: FATAL: sorry, too many clients already
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
    at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
    at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
    at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
    at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
    at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
    at org.postgresql.Driver.makeConnection(Driver.java:393)
    at org.postgresql.Driver.connect(Driver.java:267)
    at java.sql.DriverManager.getConnection(DriverManager.java:582)
    at java.sql.DriverManager.getConnection(DriverManager.java:185)
    at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:363)
    at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
    at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
    at scala.slick.jdbc.JdbcBackend$DatabaseFactoryDef$$anon$4.createConnection(JdbcBackend.scala:47)
    at scala.slick.jdbc.JdbcBackend$BaseSession.conn$lzycompute(JdbcBackend.scala:302)
    at scala.slick.jdbc.JdbcBackend$BaseSession.conn(JdbcBackend.scala:302)
    at scala.slick.jdbc.JdbcBackend$BaseSession.close(JdbcBackend.scala:316)
    at scala.slick.backend.DatabaseComponent$DatabaseDef$class.withSession(DatabaseComponent.scala:31)
    at scala.slick.jdbc.JdbcBackend$DatabaseFactoryDef$$anon$4.withSession(JdbcBackend.scala:46)
    at com.exampleapp.services.UserServiceImpl.getById(UserService.scala:37)
    at controllers.UsersController$$anonfun$show$1.apply(UsersController.scala:84)
    at controllers.UsersController$$anonfun$show$1.apply(UsersController.scala:76)
    at play.api.mvc.ActionBuilder$$anonfun$apply$10.apply(Action.scala:221)
    at play.api.mvc.ActionBuilder$$anonfun$apply$10.apply(Action.scala:220)
    at controllers.ActionWithContext$.invokeBlock(BaseController.scala:42)
    at play.api.mvc.ActionBuilder$$anon$1.apply(Action.scala:309)
    at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109)
    at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109)
    at play.utils.Threads$.withContextClassLoader(Threads.scala:18)
    at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:108)
    at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:107)
    at scala.Option.map(Option.scala:145)
    at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:107)
    at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:100)
    at play.api.libs.iteratee.Iteratee$$anonfun$mapM$1.apply(Iteratee.scala:481)
    at play.api.libs.iteratee.Iteratee$$anonfun$mapM$1.apply(Iteratee.scala:481)
    at play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1.apply(Iteratee.scala:517)
    at play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1.apply(Iteratee.scala:517)
    at play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$13.apply(Iteratee.scala:493)
    at play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$13.apply(Iteratee.scala:493)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Environment

I am running scala 2.10.3, play 2.2.3

I am launching my app using sbt run.

Postgresql driver version: 9.1-901.jdbc4
slick version: 2.0.1
bonecp version: 0.8.0.RELEASE
Blankman
  • 259,732
  • 324
  • 769
  • 1,199
  • I found [this](https://gist.github.com/filipelenfers/7914993) gist, maybe it can help you. – Ende Neu Jun 05 '14 at 17:42
  • @EndeNeu I updated my question with how I am using the db (I am using db.withSession {...}) already but thanks for that point as it makes things clearer. – Blankman Jun 05 '14 at 18:04
  • 1
    I never user Bonecp but I guess that the slick `withSession` is handled correctly, the problem is the `datSource` connection opened by Bonecp, looking at the gist you need to call `dataSource.close()` even though the slick session has expired. – Ende Neu Jun 05 '14 at 18:06
  • @EndeNeu thanks I did that I'll report back if that worked (it makes sense). So if not boncecp what do you use? – Blankman Jun 05 '14 at 18:47
  • Slick has his own jdbc driver classes, for Postgres I use `scala.slick.driver.PostgresDriver.simple._` for my models (e.g. CRUD operations) while to open a session (with `DB.withSession`) I use simply `play.api.db.slick._` and `play.api.Play.current`. I actually don't have a class that handles connections, in my entry point I simply use `DB.withSession` and pas the session around implicitly. – Ende Neu Jun 05 '14 at 18:55
  • @EndeNeu I see, so you don't use a pool of connections then right? I use the same Postgres driver as you, but I have a pool of connections. – Blankman Jun 05 '14 at 19:00
  • Ah I see now, I went to read some documentation on Bonecp. My application currently doesn't need a connection pool, although it could be useful in the future, and by the way I didn't know Slick had no support for pooling. – Ende Neu Jun 05 '14 at 19:07
  • @EndeNeu I'd like to give you credit, the .close made it work. If you want you can write an answer with that info. – Blankman Jun 09 '14 at 20:47
  • Good to know my guess was right :D – Ende Neu Jun 10 '14 at 15:47

1 Answers1

3

Slick handles session closing automatically. From the documentation:

The Database object’s withSession method creates a Session, passes it to a given function and closes it afterwards. If you use a connection pool, closing the Session returns the connection to the pool.

The problem is the Bonecp's dataSource connections which are not closed, as showed on this gist you need to close them manually using dataSource.close().

Ende Neu
  • 15,581
  • 5
  • 57
  • 68