async.waterfall inside async.each doesn't work?

Question

I'm trying to run an async.each over an array of items.

For each item, I want to run an async.waterfall. See code below.

var ids = [1, 2];

async.each(ids, 

    function (id, callback) {

        console.log('running waterfall...');

        async.waterfall([

            function (next) {

                console.log('running waterfall function 1...');

                next();
            },

            function (next) {

                console.log('running waterfall function 2...');

                next();
            }],

            function (err) {

                if (err) {
                    console.error(err);
                }
                else {
                    console.log('waterfall completed successfully!');
                }

                callback();
            });
    }, 

    function (err) {

        if (err) {
            console.error(err);
        }
        else {
            console.log('each completed successfully!');
        }

    });

return;

The output for this code looks like:

running waterfall...
running waterfall function 1...
running waterfall...
running waterfall function 1...
running waterfall function 2...
running waterfall function 2...
waterfall completed successfully!
waterfall completed successfully!
each completed successfully!

But my intention is and my understanding is that the output should look like this:

running waterfall...
running waterfall function 1...
running waterfall function 2...
waterfall completed successfully!
running waterfall...
running waterfall function 1...
running waterfall function 2...
waterfall completed successfully!
each completed successfully!

I keep looking over the code and I don't know what's wrong, does anyone know if my code or my expectations for what the async methods should be doing is incorrect?

Thank you!

jfriend00 · Accepted Answer · 2016-03-16T01:49:51.837

async.each() attempts to run all the iterations of the loop in parallel so all iterations may be in flight at the same time and will complete in an indeterminate order. You can see this clearly described in the doc for .each():

Applies the function iteratee to each item in arr, in parallel. The iteratee is called with an item from the list, and a callback for when it has finished. If the iteratee passes an error to its callback, the main callback (for the each function) is immediately called with the error.

Note, that since this function applies iteratee to each item in parallel, there is no guarantee that the iteratee functions will complete in order.

So, this explains why both your .waterfall() iterations are going at the same time rather than run serially.

If you want to run them one after the other, then you should be using async.eachSeries() instead.

Thanks @jfriend00. great answer – fooser Mar 16 '16 at 02:41 — fooser, Mar 16 '16 at 02:41

async.waterfall inside async.each doesn't work?

1 Answers1