1

I'm running StrongLoop's Loopback API server in production mode. This means that master process creates as many workers, how many cores your CPU has. So master process only controls workers and never execute your code.

My purpose is to execute periodic task just once at the time, because now it is runs on all 4 workers.

Is there any suggestions, except cron or 'key lock' in Redis-like storage?

IvanZh
  • 2,265
  • 1
  • 18
  • 26
  • Without something like Redis, you'd likely need to handle communication by sending messages between each child process in your cluster. This isn't scalable, because lets say you have 8 hosts, with 4 children on each. You'll still have 8 instances running the same task, so you're still being redundant. Maybe this is what you want, but if you scale out the only way to schedule a task to run on a single instance on a single host in your cluster is to have some kind of broke, redis being a popular option. – tsturzl Mar 28 '15 at 17:04
  • Also, you're master can execute code. So you could handle the scheduling there then broadcast the result out to each child. – tsturzl Mar 28 '15 at 17:06
  • Again, you'll still need a broker if you have multiple hosts and you only want the task to execute once. – tsturzl Mar 28 '15 at 17:07
  • With the clustering mechanism being used it is not practical to execute code in the cluster's main process because the cluster is created automagically by strong-supervisor. – Ryann Graham Mar 29 '15 at 18:44

1 Answers1

1

In iojs and node v0.12 it is possible to do an exclusive socket binding. This could be used as a form of locking similar to a filesystem based approach. The method is the same for both:

attempt exclusive access to resource
if success:
   perform task
else:
   do not perform task

With sockets you would do something like this:

net.createServer().on('error', function(err) {
  console.log('did not get lock', err);
}).listen({ port: 4321, exclusive: true }, function() {
  singleProcessTask();
  this.close();
});

Note that exclusive: true is only required for cluster mode because it defaults to sharing sockets.

Similarly with fs.open:

fs.open('lock.file', 'wx', function(err, fd) {
  if (err) {
    console.log('did not get lock', err);
  } else{
    singleProcessTask();
    fs.close(fd, function(err) {
      // insert error handling here
      fs.unlink('lock.file', function(err) {
        // insert error handling here
      });
    });
});

In both cases there is potential for race conditions if your task is very quick and your processes are on slightly different timer schedules. In these cases the task will still only be performed by one process at a time, but it may be processed multiple times per scheduled period depending on how you implement your scheduling.

edit: more illustrative example

var net = require('net');

var HOUR = 60*60*1000;

setInterval(myTask, HOUR);

function myTask() {
  locked(function() {
    // task code here
  });
}

function locked(fn) {
  net.createServer().on('error', function(err) {
    console.log('did not get lock', err);
  }).listen({ host: '127.0.0.1', port: 4321, exclusive: true }, function() {
    fn();
    this.close();
  });
}
Ryann Graham
  • 8,079
  • 2
  • 29
  • 32
  • Thank for pointing to this node feature, but is it safety to use `net` instead `http` module? Also, does running workers on different ports will help to execute `setInterval` task only at once? – IvanZh Mar 30 '15 at 18:49
  • You would only use this as a locking mechanism, you wouldn't actually be accepting connections on it. There shouldn't be much difference between using net and http for _this_ particular use. – Ryann Graham Mar 30 '15 at 19:02
  • I've added an expanded example to show how to combine the approach with setInterval. – Ryann Graham Mar 30 '15 at 19:09