Parallel Flow in Node Modules

Question

I am having an express app. I want to have parallel flow for an array of functions that I want to run. I am thinking of using async module for doing so.

I want to know if there is any other module which will be more better than this?

Secondly I want to know is it necessary that these functions be Asynchronous? Lets say I have code like this

var sum = function(x, y){
   return (x + y)
}

async.parallel([
    function(callback){
        setTimeout(function(){
            result = sum (x, y); //just an example for a synchronous function
            callback(null, result);
        }, 100);
    },
    function(callback){
        result = sum (x, y); //just an example for a synchronous function
        callback(null, result);
    }

],
// optional callback
function(err, results){
    console.log(result);
    // the results array will equal ['one','two'] even though
    // the second function had a shorter timeout.
});

SO as you can inside this there are functions which are synchronous. So will still this two run in parallel?

I have also heard that in node.js only I/O tasks can run in parallel as node.js is single threaded. Is it true? And so if I don't have I/O tasks that in async module also they won't run in parallel rather just appear to?

Please help.

xmikex83 · Answer 1 · 2013-07-14T19:42:04.047

Node.js is single-threaded and (out of the box) single-core.

So, nothing is truly parallel in Node.js since is single-threaded.

The async.parallel method is useful only if your tasks contain asynchronous code: it will wait for all the responses, then executes the code in the callback. But, if your tasks contains only synchronous code, the result will be synchronous.

So the answer to your last question is yes.

If you want to try another control flow module, try Q. It implements Javascript Promises and makes async code easier to read and to organize.

If you want to truly parallelize your tasks, look at child process or cluster module.

There are also some special native modules as Threads à gogo which implements threading.

Full example with your code:

var async = require('async');

sum = function(x, y) {
  return (x + y)
}

x = 2;
y = 3;

async.parallel([
    function(callback) {
      setTimeout(function() {
        result = sum(x, y);
        console.log('f1');
        callback(null, result);
      }, 100);
    },
    function(callback) {
      result = sum(x, y);
      console.log('f2');
      callback(null, result);
    }

  ],

  function(err, results) {
    console.log(result);
  }
);

If you run this piece of code output result will always be: f2, f1, 5. This is because of setTimeout in the first function, but the way the code is executed is synchronous.

If you want to be convinced, remove setTimeout in the first function and see that output will always be f1, f2, 5.

So:

async.parallel is very useful if you want to run two HTTP requests (which are asynchronous) at the same time, and wait for both of them to be completed.
async.parallel is not useful if you put synchronous code in it, because the code will be executed in a synchronous way.

I have listed the kind of code I am going to run in the question. Take a look at it and let me know if those can run in parallel mode. As it seems to me, they are asynchronous but have synchronous function inside them. So please answer if those can run in parallel. — Saransh Mohapatra, Jul 14 '13 at 19:17

score 1 · Answer 2 · answered Jul 14 '13 at 18:57

1

async is the de facto module for these kind of tasks, go with it.

No, it is not mandatory for those function to be asynchronous. I belive all your questions could be answered with this: http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/, where the following is the important point:

Node.js keeps a single thread for your code however, everything runs in parallel except your code.

answered Jul 14 '13 at 18:57

Alberto Zaccagni

30,779
11
72
106

Are you sure that they don't need to be synchronous? And secondly will non I/O also work parallel or just work like this. As many have told me that due to node.js running in single thread its not possible, and to achieve it have to spawn child processes. So do async use child process for the same? – Saransh Mohapatra Jul 14 '13 at 19:01
The callback function of `async.parallel` will be called when both functions are done with their work (ie when they call their callback), this has nothing to do with the nature of the functions involved. No, async doesn't spawn processes to achieve that, read the link and just the `async.parallel` to get a precise idea. – Alberto Zaccagni Jul 14 '13 at 19:10
Yeah I went through both but couldn't find an answer as to how async.parallel achieves this? So was curious to know how does it do so? – Saransh Mohapatra Jul 14 '13 at 19:14
It implements the functionality with a counter. https://github.com/caolan/async/blob/master/lib/async.js#L101-L122 – Alberto Zaccagni Jul 14 '13 at 19:26
Are you sure about the synchronous functions can also be run thing? – Saransh Mohapatra Jul 14 '13 at 20:29
You could always try it :) – Alberto Zaccagni Jul 14 '13 at 20:35

score 0 · Answer 3 · answered Jun 20 '16 at 10:44

Another way is to use promise (bluebird). See doc and sample usage here http://bluebirdjs.com/docs/getting-started.html).

e.g.,

function f1() {... }
function f2() {... }
function f3() {... }

Promise.all[f1(), f2(), f3()] // run all functions parallel
  .then(() => { /*do something*/})
  .catch(err => { /*handle error*/ })

Parallel Flow in Node Modules

3 Answers3