Mongo Connection is created multiple times in RESTful API and never released

Question

I have written a RESTful API using Apache Jersey. I am using MongoDB as my backend. I used Morphia (v.1.3.4) to map and persist POJO to database. I tried to follow "1 application 1 connection" in my API as recommended everywhere but I am not sure I am successful. I run my API in Tomcat 8. I also ran Mongostat to see the details and connection. At start, Mongostat showed 1 connection to MongoDB server. I tested my API using Postman and it was working fine. I then created a load test in SoapUI where I simulated 100 users per second. I saw the update in Mongostat. I saw there were 103 connections. Here is the gif which shows this behaviour.

I am not sure why there are so many connections. The interesting fact is that number of mongo connection are directly proportional to number of users I create on SoapUI. Why is that? I found other similar questions but I think I have implemented there suggestions.

Mongo connection leak with morphia

Spring data mongodb not closing mongodb connections

My code looks like this.

DatabaseConnection.java

// Some imports
public class DatabaseConnection {

    private static volatile MongoClient instance;
    private static String cloudhost="localhost";

    private DatabaseConnection() { }

    public synchronized static MongoClient getMongoClient() {
        if (instance == null ) {
            synchronized (DatabaseConnection.class) {
                if (instance == null) {
                    ServerAddress addr = new ServerAddress(cloudhost, 27017);
                    List<MongoCredential> credentialsList = new ArrayList<MongoCredential>();
                    MongoCredential credentia = MongoCredential.createCredential(
                        "test", "test", "test".toCharArray());
                    credentialsList.add(credentia);
                    instance = new MongoClient(addr, credentialsList); 

                }
            }
        }
        return instance;
    }
}

PourService.java

@Secured
@Path("pours")
public class PourService {

final static Logger logger = Logger.getLogger(Pour.class);
private static final int POUR_SIZE = 30;

    @POST
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    public Response createPour(String request)
    {
        WebApiResponse response = new WebApiResponse();
        Gson gson = new GsonBuilder().setDateFormat("dd/MM/yyyy HH:mm:ss").create();
        String message = "Pour was not created.";
        HashMap<String, Object> data = null;
        try
        {
            Pour pour = gson.fromJson(request, Pour.class);

            // Storing the pour to 
            PourRepository pourRepository = new PourRepository();           
            String id = pourRepository.createPour(pour);

            data = new HashMap<String, Object>();
            if ("" != id && null != id)
            {
                data.put("id", id);
                message = "Pour was created successfully.";
                logger.debug(message);
                return response.build(true, message, data, 200);
            }

            logger.debug(message);
            return response.build(false, message, data, 500);
        }
        catch (Exception e)
        {
            message = "Error while creating Pour.";
            logger.error(message, e);
            return response.build(false, message, new Object(),500);
        }
    }

PourDao.java

public class PourDao extends BasicDAO<Pour, String>{

    public PourDao(Class<Pour> entityClass, Datastore ds) {
        super(entityClass, ds);
    }
}

PourRepository.java

public class PourRepository {

    private PourDao pourDao;

    final static Logger logger = Logger.getLogger(PourRepository.class);

    public PourRepository ()
    {
        try 
        {
            MongoClient mongoClient = DatabaseConnection.getMongoClient();
            Datastore ds = new Morphia().map(Pour.class)
                    .createDatastore(mongoClient, "tilt45");
            pourDao = new PourDao(Pour.class,ds);
        } 
        catch (Exception e) 
        {
            logger.error("Error while creating PourDao", e);
        }
    }

    public String createPour (Pour pour)
    {
        try
        {
            return pourDao.save(pour).getId().toString();
        }
        catch (Exception e)
        {
            logger.error("Error while creating Pour.", e);
            return null;
        }
    } 
 }

There's nothing particularly surprising or concerning about your observations. You're spawning 100 threads which are in turn making 100 connections to the database. — helmy, Jul 10 '17 at 17:56
Do the number of connections reach a steady state (i.e. ~100)? The MongoDB Java driver has a default connection pool size of 100. Try varying this with the [`maxPoolSize` option](http://mongodb.github.io/mongo-java-driver/3.2/driver/reference/connecting/connection-settings/#options). I suspect the 100 connections include the default connection pool plus a few connections for the `mongo` shell and `mongostat` commands you are running. — Stennie, Jul 12 '17 at 13:36
@helmy 1 user = 1 mongo connection is not a right way. This is not scalable. — Darshan Puranik, Jul 18 '17 at 10:44
I didn't say that and your test doesn't prove that. Try your test again and simulate 200 users, you will most likely see similar results, e.g. ~100 connections. — helmy, Jul 18 '17 at 15:45

score 1 · Answer 1 · edited Jul 13 '17 at 22:33

When I work with Mongo+Morphia I get better results using a Factory pattern for the Datastore and not for the MongoClient, for instance, check the following class:

public DatastoreFactory(String dbHost, int dbPort, String dbName) {
    final Morphia morphia = new Morphia();
    MongoClientOptions.Builder options = MongoClientOptions.builder().socketKeepAlive(true);
    morphia.getMapper().getOptions().setStoreEmpties(true);
    final Datastore store = morphia.createDatastore(new MongoClient(new ServerAddress(dbHost, dbPort), options.build()), dbName);
    store.ensureIndexes();
    this.datastore = store;
}

With that approach, everytime you need a datastore you can use the one provided by the factory. Of course, this can implemented better if you use a framework/library that support factory pattern (e.g.: HK2 with org.glassfish.hk2.api.Factory), and also singleton binding.

Besides, you can check the documentation of MongoClientOptions's builder method, perhaps you can find a better connection control there.

My datastore is mapped concrete class "Pour" and many other classes in different file. Hence I created singleton pattern at MongoClient level. According to documentation MongoClient is managing connection for application and there should be only one instance of MongoClient per application. — Darshan Puranik, Jul 18 '17 at 08:48

Mongo Connection is created multiple times in RESTful API and never released

1 Answers1