0

We use MongoDB with Parse-server as backend and our application queries and save data at a rate of 5 request/second.

And out of our 3-stack solution (Nginx, Node.js, MongoDB), MongoDB takes the highest CPU hit, perhaps due to the query and save operation, we are using Jelastic so what we do is to pump-up the CPU resources available for our server, but everytime MongoDB CPU usage is keeping up.

I think this can be attributed to the fact that we implemented a check if a document with the same field value exists before saving and after saving doing a dirty check of duplicate record and remove the last record that is a duplicate (just keeping the oldest one) at Node.js level.

enter image description here

The question now would be:

  1. Would configuring MongoDB Replica set help to reduce CPU usage?
  2. What could be done at MongoDB level to optimize it to be able to handle such process/request describe above?

Here's the code:

Parse.Cloud.beforeSave("ProcessedDocument", function(request, response) {
    var d = request.object;
    var documentId = d.get("documentId");
    var query = new Parse.Query("ProcessedDocument");
    query.equalTo("documentId", documentId);
    query.first({
        success: function(results) {
            //console.log('Results ' + results);
            if(results) {
                if (!request.object.isNew()) {
                    response.success();
                } else {
                    response.error({errorCode:400,errorMsg:"Document already exist"});
                }
            } else {
                response.success();
            }
        },
        error: function(error) {
            response.success();
        }
    });
});
Parse.Cloud.afterSave("ProcessedDocument", function(request) {
    var query = new Parse.Query("ProcessedDocument");
    query.equalTo("documentId", request.object.get("documentId"));
    query.ascending("createdAt");
    query.find({
        success:function(results) {
            if (results && results.length > 1) {
                for(var i = (results.length - 1); i > 0 ; i--) {
                    results[i].destroy();
                }
            }
            else {
                // No duplicates
            }
        },
        error:function(error) {
        }
    });
});

Here's the performance snapshot from MongoDB Compass:

enter image description here

quarks
  • 33,478
  • 73
  • 290
  • 513
  • Shouldn't this be the OS'es job? Typically the applications consume as much time as they get assigned. What about using ``nice`` or ``cpulimit`` on *nix boxes? – Jan B. Mar 22 '18 at 17:02
  • 1. no. it will do the opposite. 2. it depends what's causing high CPU usage. As a side note - I can't believe mongodb uses less than a half-GB of RAM. – Alex Blex Mar 22 '18 at 17:04
  • Based on MongoDB compass, the CPU usage of MongoDB that our platform hosting reports is mainly attributed to AREADS – quarks Mar 22 '18 at 17:57

0 Answers0