Access same setTimeout() instance from multiple Node.js instances

Question

Our API needs to send data to Zapier if some specific data was modified in our DB.

For example, we have a company table and if the name or the address field was modified, we trigger the Zapier hook.

Sometimes our API receives multiple change requests in a few minutes, but we don't want to trigger the Zapier hook multiple times (since it is quite expensive), so we call a setTimeout() (and overwrites the existing setTimeout) on each modify requests , with a 5000ms delay.

It works fine, and there are no multiple Zapier hook calls even if we get a lot modify requests from client in this 5000ms period.

Now - since our traffic is growing - we'd like to set up multiple node.js instances behind some load balancer.

But in this case the different Node.js instances can not use - and overwrite - the same setTimeout instance, which would cause a lot useless Zapier calls.

Could you guys help us, how to solve this problem - while remaining scalable?

Juan Stiza · Accepted Answer · 2015-12-17T12:41:41.250

If you want to keep a state between separate instances you should consider, from an infrastructure point of view, some locking mechanism such as Redis. Whenever you want to run the Zapier call, if no lock is active, you set one on Redis, all other calls won't be triggered as it is locked, whenever the setTimeout callback runs, you disable the Lock. Beware that Redis might become a SPOF, I don't know where you are hosting your services, but that might be an important point to consider.

Edit:

The lock on Redis might have a reference to the last piece of info you want to update. So on the first request you set the data to be saved on Redis, wait 5 seconds, and update. If any modifications were made in that time frame, it will be stored on Redis, that way you'll only update on 5 second intervals, you'll need to add some extra logic here though. Example:

function zapierUpdate(data) {

  if (isLocked()) {

    // Locked! We will update the data that needs to be saved on the
    // next setTimeout callback
    updateLockData(data);

  } else {

    // First lock and save data.
    lock(data);
    // and update in 5 seconds
    setTimeout(function(){
      // getLockData fetches the data on Redis and releases the lock
      var newData = getLockData();
      // Update the latest data that might have been updated.
      callZapierNow(newData);
    },5000);

  }

}

Access same setTimeout() instance from multiple Node.js instances

1 Answers1