Why multiple mongodb connecions with Casbah?

Question

I have to manage multiple databases connection to MongoDb, using casbah scala client. I have an approximation that works but open hundreds of connections.

I want to save a Map[String, MongoDB] that saves a connection for each database (which is the key. I'm using this in Spark Streaming with a two nodes cluster, so I think that is a serialization issue but I don't know how to fix it.

Take a look to my class.

abstract class AbstractMongoDAO(@transient val config: Config) extends Closeable with Serializable {

      @transient private val mongoConfig = config.getConfig(CONFIG_KEY)
      private val host = mongoConfig.getString(CONFIG_KEY_HOST)

      @transient private var _mongoClient: MongoClient = MongoClient(host)
      private var _dbs: mutable.HashMap[String, MongoDB] = mutable.HashMap()

      protected  def dbs(): mutable.HashMap[String, MongoDB] ={
        if(_dbs == null)
          _dbs = mutable.HashMap()
        _dbs
      }

      def mongoClient: MongoClient = {
        if (_mongoClient == null) {
          _mongoClient = MongoClient(host)
        }
        _mongoClient
      }

      def db(dbName: String):MongoDB =  {
        if (dbs.get(dbName) == None) {
          _dbs += (dbName ->  mongoClient.getDB(dbName))
        }
        _dbs.get(dbName).get
      }

      override def close() = {
        Option(_mongoClient).foreach(_.close())
      }

    }

    private object AbstractMongoDAO {
      val CONFIG_KEY = "mongo"
      val CONFIG_KEY_HOST = "host"

    }

And then I have another class that extends AbstractMongoDao

class MongoDAO (override val config : Config)
  extends AbstractMongoDAO(config) with Serializable

And I get a db connection with this simple code. appName is a variable database.

val _db = db(appName)

What I'm doing wrong?

score 4 · Answer 1 · answered Sep 14 '14 at 06:42

Casbah is built on top of official Java driver. MongoClient represents an internal pool of db connections to a MongoDB cluster. If you use the same cluster and only change database name and not the host, you don't need to create multiple MongoClients, one would be enough for the whole application.

To configure MongoClient check this documentation and corresponding options. If you have multiple DB hosts or still want to use multiple MongoClients then you can build your options and create MongoClient like this:

val options = MongoClientOptions.builder()
            .connectionsPerHost(1)
            // add other options if needed
            .build();

 val _mongoClient = MongoClient(host, options)

In your case since only db name neeeds to change and not the db host I would change the method that gets db to this:

  def db(dbName: String):MongoDB =
    mongoClient.getDB(dbName) // db will be created in Mongo on the fly if not exist

And you don't need the map anymore.

I did what you told me but still creates multiple connections... I put connections in a companion object with your mongoclientoptions and now works correctly. Thank you. — gasparms, Sep 15 '14 at 08:42

Why multiple mongodb connecions with Casbah?

1 Answers1