NodeJS cluster, Is it really needed?

Question

I decided that i want to investigate what is the best possible way to handle big amount of traffic with NodeJS server, i did a small test on 2 digital ocean servers which has 1GB RAM / 2 CPUs No-Cluster server code:

// Include Express
var express = require('express');

// Create a new Express application
var app = express();

// Add a basic route – index page
app.get('/', function (req, res) {
    res.redirect('http://www.google.co.il');
});

// Bind to a port
app.listen(3000);
console.log('Application running');

Cluster server code:

    // Include the cluster module
var cluster = require('cluster');
// Code to run if we're in the master process
if (cluster.isMaster) {
    // Count the machine's CPUs
    var cpuCount = require('os').cpus().length;

    // Create a worker for each CPU
    for (var i = 0; i < cpuCount; i += 1) {
        cluster.fork();
    }
// Code to run if we're in a worker process
} else {
    // Include Express
    var express = require('express');

    // Create a new Express application
    var app = express();

    // Add a basic route – index page
    app.get('/', function (req, res) {
        res.redirect('http://www.walla.co.il');
    });

    // Bind to a port
    app.listen(3001);
    console.log('Application running #' + cluster.worker.id);
}

And i sent stress test requests to those servers, i excepted that the cluster server will handle more requests but it didn't happen, both servers crashed on the same load, although 2 node services were running on the cluster and 1 service on the non-cluster.

Now i wonder why ? Did i do anything wrong?

Maybe something else is making the servers reach its breakpoint? both servers crashed at ~800 rps

Was there a total of 2 node processes or a total of 3 node processes ?, because including master, you will need 3 node processes in the cluser — Malice, Jan 01 '18 at 14:36

score 12 · Accepted Answer · answered Jan 01 '18 at 14:59

Now i wonder why ? did i do anything wrong?

Your test server doesn't do anything other than a res.redirect(). If your request handlers use essentially no CPU, then you aren't going to be CPU bound at all and you won't benefit from involving more CPUs. Your cluster will be bottlenecked at the handling of incoming connections which is going to be roughly the same with or without clustering.

Now, add some significant CPU usage to your request handler and you should get a different result.

For example, change to this:

// Add a basic route – index page
app.get('/', function (req, res) {

    // spin CPU for 200ms to simulate using some CPU in the request handler
    let start = Date.now();
    while (Date.now() - start < 200) {}

    res.redirect('http://www.walla.co.il');
});

Running tests is a great thing, but you have to be careful what exactly you're testing.

score 1 · Answer 2 · edited Jun 20 '20 at 09:12

What @jfriend00 says is correct; you aren't actually doing enough heavy lifting to justify this, however, you're not actually sharing the load. See here:

app.listen(3001);

You can't bind two services onto the same port and have the OS magically load-balance them[1]; try adding an error handler on app.listen() and see if you get an error, e.g.

app.listen(3001, (err) => err ? console.error(err));

If you want to do this, you'll have to accept everything in your master, then instruct the workers to do the task, then pass the results back to the master again.

It's generally easier not to do this in your Node program though; your frontend will still be the limiting factor. An easier (and faster) way may be to put a special purpose load-balancer in front of multiple running instances of your application (i.e. HAProxy or Nginx).

[1]: That's actually a lie; sorry. You can do this by specifying SO_REUSEPORT when doing the initial bind call, but you can't explicitly specify that in Node, and Node doesn't specify it for you...so you can't in Node.

NodeJS cluster, Is it really needed?

2 Answers2