Use MongoDb Connection pooling with Go Application

Question

I am working on a Golang SAAS application along with Mongodb. Previously I worked with db connections without pooling. As a result of this, my db hangs on or shuts down when some traffic comes in.

Then I get to know about connection pooling. I explored it but I have some doubts whether it will go with my application structure or not.

I am providing here some code samples from my application.

Function to create connection to db:

func ConnectDb(merchantDb string) (mongoSession *mgo.Session) {
    mongoDBDialInfo := &mgo.DialInfo{
        Addrs:     []string{DatabaseIpPort},
        Username:  DbUsername,
        Password:  DbPassword,
        Source:    DbSource,
        Database:  merchantDb,
        Timeout:   60 * time.Second,
        PoolLimit: 4096,
    }
    mongoSession, err := mgo.DialWithInfo(mongoDBDialInfo)
    if err != nil {
        fmt.Printf("CreateSession: %s\n", err)
        defer mongoSession.Close()
        return mongoSession
    }
    mongoSession.SetMode(mgo.Monotonic, true)
    return mongoSession
}

Example of a model function which connects to db:

func (MerchantDb *MerchantDatabase) UpdateCustomer(uid int, query interface{}) (err error) {
    mongoSession := config.ConnectDb(MerchantDb.Database)
    defer mongoSession.Close()

    sessionCopy := mongoSession.Copy()
    defer sessionCopy.Close()

    getCollection := sessionCopy.DB(MerchantDb.Database).C("customers")
    err = getCollection.Update(bson.M{"uid": uid}, query)
    return err
}

How I call this function:

type MerchantDatabase struct {
    Database string
}
merchantDb := MerchantDatabase{c.Keys["merchant_db"].(string)}
merchantDb.UpdateUser(7, bson.M{"$set": bson.M{"name": "john"})

Like above code I use different model function for every query & in every model function a new connection is established with mongodb.

My queries are:

1. I just passed PoolLimit with the existing code. Is this the way connection pool works? Or there something else?
2. Also in my application, there are multiple databases according to merchant. Thats why I call every model function with database signature, so that it can query on that particular database. I am confused about as connection pool works as a cache of connections, can it use one merchant's database connection for other merchant's database ? If yes, how can I prevent this with connection pooling ?
3. How can I decide the limit for connection pool for my application ?

I will very grateful to receive the replies.

Thanks!

Answer 1

The problem is that you create a new connection pool in each UpdateCustomer's call, this is inconvenient and costly.

You can use inversion of control:

• Prepare a connexion pool in the application setup.
• Inject the prepared mgo session in your model’s struct (e.g MerchantDatabase)
• Use the injected session in your model's function calls to connect to the corresponding merchant’s DB.

Note: I suppose that all merchant’s DBs are hosted in the same mongo server node or replica set.

Answer 2

I'm guessing this was copied & pasted from somewhere without actually understanding what it does or why it would be used:

func (MerchantDb *MerchantDatabase) UpdateCustomer(uid int, query interface{}) (err error) {
    mongoSession := config.ConnectDb(MerchantDb.Database)
    defer mongoSession.Close()

    sessionCopy := mongoSession.Copy()
    defer sessionCopy.Close()

    getCollection := sessionCopy.DB(MerchantDb.Database).C("customers")
    err = getCollection.Update(bson.M{"uid": uid}, query)
    return err
}

mongoSession.Copy() is how you would get a new session and correctly use pooling. mongoSession is effectively the pool. But here, you're creating a connection, creating another connection, doing your query, then closing both connections. No pooling.

If you look at any tutorials for mgo, what you'll find is that you should create one main connection to act as your pool, and when you need a connection from the pool, you Copy() that main connection, use the copy as needed, then close the copy. You only close the original when the application exits (generally speaking). The original is used as a factory to create pooled connections from.

When I Googled "go mongo mgo connection pool", I got these top results that explain how to do it:

https://www.ardanlabs.com/blog/2014/02/running-queries-concurrently-against.html

Connections pool in Go mgo package

However this is something of a moot point - as the mgo documentation and readme both state, it has been deprecated and is no longer maintained. There is now an official driver from Mongo: https://github.com/mongodb/mongo-go-driver