How to use a for loop and get back results using node?

Question

I am trying to run a script that does the same thing multiple times, but I don't want to get stuck writing callback after callback. I am a bit of a beginner with node.js, so I am not sure the proper way of doing this. Here is my code below:

exports.index = function (req,res) {
  /* some code */
  something.execute(this, that, function(results) {
    for (var i = 2; i <= 10; i++) {
      var nextPage = getNextPage(i);
    }
  }
}

function getNextPage(page) {
  something.execute(this, that, function(results) {
    return results;
  }
}

What happens is that result returns as undefined. I know that this has to do something to do with that getNextPage is completed after the rest of the code because node is asynchronous, but I am unsure what the best way to approach this is.

Any suggestions? Thanks!

Edit: Added code in getNextPage. I am pretty much running the same code for 9 more pages. It seems as though I need to figure out how to force the something.execute loop to run.

Also, we can't say what's wrong until you show how you are assigning `result`. — Raj, Mar 02 '14 at 14:00
you need to show more code or explain a bit more what your trying to do. if you suggest getNextPage() return undefined then you need to show the code for getNextPage() — wayne, Mar 02 '14 at 14:01
It sounds like you know you can solve this problem by adding another callback, but you're reluctant to go there. A common pattern for combatting "too many nested callbacks" is to use any of several "Promises" libraries. https://github.com/kriskowal/q is one such. You might want to check it out. Promises add a bit of boilerplate to your async calls, but decouple them from your callbacks. — Jason Reid, Mar 02 '14 at 14:25
I agree with Jason's promises library suggestion as a general way to avoid the nested callback blues, which I think you will find otherwise unavoidable (nature of the node beast). If it is may be helpful, I provided an [answer](http://stackoverflow.com/questions/22109487/nodejs-mysql-dump/22110015#22110015) with a heavily-commented Q promises example yesterday as well as the equivalent non-promise version. More generally, the other commenters are correct that we'd need more code to provide any real help. — barry-johnson, Mar 02 '14 at 14:34
+1 for using promises for flow control. Although for paged information using a stream would be my preferred approach. https://www.npmjs.org/package/event-stream — Scott Puleo, Mar 02 '14 at 14:43
I just looked up the Q library, but it is quite confusing to me. I added some more code in my question, would it be the right tool? — David Mckee, Mar 02 '14 at 20:57

score 1 · Answer 1 · answered Mar 02 '14 at 15:34

1

The best overview of the many ways/styles of coding this in node I think is Mixu's Node Book Ch 7 "Control Flow". Specifically examples "7.2.1 Control flow pattern #1: Series - an asynchronous for loop" and "7.2.2 Control flow pattern #2: Full parallel - an asynchronous, parallel for loop". I pasted 7.2.1 here for you since many stackoverflow users frown upon external links without an excerpt.

function async(arg, callback) {
  console.log('do something with \''+arg+'\', return 1 sec later');
  setTimeout(function() { callback(arg * 2); }, 1000);
}
function final() { console.log('Done', results); }

var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];

items.forEach(function(item) {
  async(item, function(result){
    results.push(result);
    if(results.length == items.length) {
      final();
    }
  })
})

answered Mar 02 '14 at 15:34

Peter Lyons

142,938
30
279
274

This may work, but it's quite different semantically: it executes `forEach` items in parallel. Back to the OP's code, it would be a problem if the asynchronous `getNextPage` depends on the result of the previous call. The `async.forEach` from Async.js would be more appropriate, IMO. It executes items asynchronously, but sequentially. – noseratio Mar 02 '14 at 23:58
Yes, indeed. My point is in order to write node programs, you have to know the entire topic. Thus the link to the book. There's only so many times I want to post an answer for folks hitting the async challenge of javascript for the first time. I personally would probably use `async.eachLimit` but the message I'm trying to send is this is a core thing you just have to understand. – Peter Lyons Mar 03 '14 at 00:17
I see. I'm really looking forward for across-the-board, production support for `yield` in JavaScript, both in Node and all major browsers. It would make asynchronous programming in JavaScript a breeze, same as it is with `async/await` in C#. – noseratio Mar 03 '14 at 00:24

score 1 · Answer 2 · answered Mar 02 '14 at 22:18

Try the async module. async provides a number of convenient asynchronous control flow functions. You'll have the skill and confidence to write your own, later. For now, though:

var async = require('async');

function requestHandler(req, res) {
    var itemNumbers = ... // might depend on req; might not

    function getItem(itemNumber, callback) {
        // ... asynchronous function which calls back with (err, result)
    }

    function withResults(err, results) {
        if (err) {
            // handle the error
        } else {
            // respond normally
        }
    }

    async.map(itemNumbers, async, withResults);
}

async.map will call getItem once per item in itemNumbers. If any invocation getItem calls back with an error, async.map will call withResults back with that error. Otherwise, async.map will call back with a null for err and an array of getItem callback results for results.

How to use a for loop and get back results using node?

2 Answers2