We use MongoDB with Parse-server as backend and our application queries and save data at a rate of 5 request/second.
And out of our 3-stack solution (Nginx, Node.js, MongoDB), MongoDB takes the highest CPU hit, perhaps due to the query and save operation, we are using Jelastic so what we do is to pump-up the CPU resources available for our server, but everytime MongoDB CPU usage is keeping up.
I think this can be attributed to the fact that we implemented a check if a document with the same field value exists before saving and after saving doing a dirty check of duplicate record and remove the last record that is a duplicate (just keeping the oldest one) at Node.js level.
The question now would be:
- Would configuring MongoDB Replica set help to reduce CPU usage?
- What could be done at MongoDB level to optimize it to be able to handle such process/request describe above?
Here's the code:
Parse.Cloud.beforeSave("ProcessedDocument", function(request, response) {
var d = request.object;
var documentId = d.get("documentId");
var query = new Parse.Query("ProcessedDocument");
query.equalTo("documentId", documentId);
query.first({
success: function(results) {
//console.log('Results ' + results);
if(results) {
if (!request.object.isNew()) {
response.success();
} else {
response.error({errorCode:400,errorMsg:"Document already exist"});
}
} else {
response.success();
}
},
error: function(error) {
response.success();
}
});
});
Parse.Cloud.afterSave("ProcessedDocument", function(request) {
var query = new Parse.Query("ProcessedDocument");
query.equalTo("documentId", request.object.get("documentId"));
query.ascending("createdAt");
query.find({
success:function(results) {
if (results && results.length > 1) {
for(var i = (results.length - 1); i > 0 ; i--) {
results[i].destroy();
}
}
else {
// No duplicates
}
},
error:function(error) {
}
});
});
Here's the performance snapshot from MongoDB Compass: