Node cluster: Handle task only to one worker

Question

I´m trying to set up cluster to multi-thread my app but instead of delegating one task per worker each task is being delegated to all workers.

My code goes as follows:

//main.js
var cluster = require('cluster');

//Code executed by the master process.
if (cluster.isMaster) {
    var numWorkers = require('os').cpus().length;
    console.log('Setting up ' + numWorkers + ' workers.');

    for (var i = 0; i< numWorkers; i++) {
        cluster.fork();
    }

    cluster.on('online', function(worker) {
        console.log('Worker ' + worker.process.pid + ' is online');
    });

    //Spawn a new worker on unexpected events.
    cluster.on('exit', function(worker, code, signal) {
        console.log('Worker ' + worker.process.pid + ' died with code: ' + code + ', and signal: ' + signal);
        console.log('Starting a new worker');
        cluster.fork();
    });
} else {
    //Set up DB
    var pg = require ('pg');
    var db = 'tcp://username:password@localhost/dbname';

    var pg_client = new pg.Client(db);
    pg_client.connect();

    //Set up Listener
    var query = pg_client.query('LISTEN newevent');
    pg_client.on('notification', function(not) {
        console.log(not.payload + ' added to queue by ' + process.pid);
    })
}

Every worker is processing the content of the else block so each time I insert a value into my table (a trigger fires up and notifies my pg_client who is listening) I get as many console.logs as workers cluster.isMaster set up however I´d like that only one worker would process each postgres event. How could I fix my code to achieve said behavior?

You would need a way of locking the task so that the first worker that gets it will somehow cause the others to instead exit early. though.... this sounds like it would be very vulnerable to race conditions. — Kevin B, Oct 16 '15 at 19:03

score 3 · Accepted Answer · answered Oct 16 '15 at 19:08

3

This would then not require the use of cluster. Cluster distributes processing evenly.

You need to change the if-else-logic then entirely because...

for (var i = 0; i< numWorkers; i++) {
     cluster.fork();
}

...expresses explicitly that you want N-Processes. Rather then just child_process.fork(); which would execute explicitly not on the main thread. Read up on it here.

answered Oct 16 '15 at 19:08

eljefedelrodeodeljefe

6,304
7
29
61

You were right, that was not the correct use case for cluster, `child_process.fork();` does seem like a better choice, Thanks! – Yonson Oct 16 '15 at 20:49
upvoted! what if you wanted both, I want 4 worker processes to handle express requests with socketio but at the same time, i have 4 unequal tasks that I want to run on each worker like task1 on worker 1, task 2 on worker 2 etc... – PirateApp Nov 13 '18 at 04:24
1

@PirateApp 2 years late, but similar situation... I spawn multiple instances of my app with PM2, so I identify master process and start my RabbitMQ consumer worker there. Now I've locked exclusive access to the mail queue on assert, because I'm certain I will only ever have one consumer running in the entire cluster. – dystopiandev Apr 10 '20 at 16:04

Node cluster: Handle task only to one worker

1 Answers1