Node.js Asynchronous Library Comparison - Q vs Async

Question

I have used kriskowal's Q library for a project (web scraper / human-activity simulator) and have become acquainted with promises, returning them and resolving/rejecting them, and the library's basic asynchronous control flow methods and error-throwing/catching mechanisms have proven essential.

I have encountered some issues though. My promise.then calls and my callbacks have the uncanny tendency to form pyramids. Sometimes it's for scoping reasons, other times it's to guarantee a certain order of events. (I suppose I might be able to fix some of these problems by refactoring, but going forward I want to avoid "callback hell" altogether.)

Also, debugging is very frustrating. I spend a lot of time console.log-ing my way to the source of errors and bugs; after I finally find them I will start throwing errors there and catching them somewhere else with promise.finally, but the process of locating the errors in the first place is arduous.

Also, in my project, order matters. I need to do pretty much everything sequentially. Oftentimes I find myself generating arrays of functions that return promises and then chaining them to each other using Array.prototype.reduce, which I don't think I should have to do.

Here is an example of one of my methods that uses this reduction technique:

removeItem: function (itemId) {

  var removeRegexp = new RegExp('\\/stock\\.php\\?remove=' + itemId);

  return this.getPage('/stock.php')
  .then(function (webpage) {
    var
      pageCount = 5,
      promiseFunctions = [],
      promiseSequence;

    // Create an array of promise-yielding functions that can run sequentially.
    _.times(pageCount, function (i) {
      var promiseFunction = function () {
        var
          promise,
          path;

        if (i === 0) {
          promise = Q(webpage);
        } else {
          path = '/stock.php?p=' + i;
          promise = this.getPage(path);
        }

        return promise.then(function (webpage) {
          var
            removeMatch = webpage.match(removeRegexp),
            removePath;

          if (removeMatch !== null) {
            removePath = removeitemMatch[0];

            return this.getPage(removePath)
            .delay(1000)
            // Stop calling subsequent promises.
            .thenResolve(true);
          }

          // Don't stop calling subsequent promises.
          return false;

        }.bind(this));
      }.bind(this);

      promiseFunctions.push(promiseFunction);
    }, this);

    // Resolve the promises sequentially but stop early if the item is found.
    promiseSequence = promiseFunctions.reduce(function (soFar, promiseFunction, index) {
      return soFar.then(function (stop) {
        if (stop) {
          return true;
        } else {
          return Q.delay(1000).then(promiseFunction);
        }
      });
    }, Q());

    return promiseSequence;
  }.bind(this))
  .fail(function (onRejected) {
    console.log(onRejected);
  });
},

I have other methods that do basically the same thing but which are suffering from much worse indentation woes.

I'm considering refactoring my project using coalan's async library. It seems similar to Q, but I want to know exactly how they differ. The impression I am getting is that async more "callback-centric" while Q is "promise-centric".

Question: Given my problems and project requirements, what would I gain and/or lose by using async over Q? How do the libraries compare? (Particularly in terms of executing series of tasks sequentially and debugging/error-handling?)

Answer 1

Both libraries are good. I have discovered that they serve separate purposes and can be used in tandem.

Q provides the developer with promise objects, which are future representations of values. Useful for time travelling.

Async provides the developer with asynchronous versions of control structures and aggregate operations.

An example from one attempt at a linter implementation demonstrates a potential unity among libraries:

function lint(files, callback) {

    // Function which returns a promise.
    var getMerged = merger('.jslintrc'),

        // Result objects to invoke callback with.
        results = [];

    async.each(files, function (file, callback) {
        fs.exists(file, function (exists) {

            // Future representation of the file's contents.
            var contentsPromise,

                // Future representation of JSLINT options from .jslintrc files.
                optionPromise;

            if (!exists) {
                callback();
                return;
            }

            contentsPromise = q.nfcall(fs.readFile, file, 'utf8');
            optionPromise = getMerged(path.dirname(file));

            // Parallelize IO operations.
            q.all([contentsPromise, optionPromise])
                .spread(function (contents, option) {
                    var success = JSLINT(contents, option),
                        errors,
                        fileResults;
                    if (!success) {
                        errors = JSLINT.data().errors;
                        fileResults = errors.reduce(function (soFar, error) {
                            if (error === null) {
                                return soFar;
                            }
                            return soFar.concat({
                                file: file,
                                error: error
                            });
                        }, []);
                        results = results.concat(fileResults);
                    }
                    process.nextTick(callback);
                })
                .catch(function (error) {
                    process.nextTick(function () {
                        callback(error);
                    });
                })
                .done();
        });
    }, function (error) {
        results = results.sort(function (a, b) {
            return a.file.charCodeAt(0) - b.file.charCodeAt(0);
        });
        callback(error, results);
    });
}

I want to do something potentially-blocking for each file. So async.each is the obvious choice. I can parallelize related operations per-iteration with q.all and reuse my option values if they apply to 2 or more files.

Here, Async and Q each influence the control flow of the program, and Q represents values resolving to file contents sometime in the future. The libraries work well together. One does not need to "choose one over the other".

Answer 2

Callback pyramids in your code can be simplified using promise composition and javascript lexical scoping.

removeItem: function (itemId) {

  var removeRegexp = new RegExp('\\/stock\\.php\\?remove=' + itemId);
  var found = false
  var promise = getPage('/sock.php')

  _.times(5, (i) => {
    promise = promise.then((webpage) => {
      if (found) return true
      var removeMatch = webpage.match(removeRegexp)
      var found = removeMath !== null
      var nextPage = found ? removeMatch[0] : '/stock.php?p='+i+1
      return Q.delay(1000).then(() => this.getPage(nextPage))
    })
  })

  return promise.fail(console.log.bind(console))

},

IMHO async should not be used in new javascript code. Promises are more composable, and allow for a lot more intutive code.

The primary reason why node did not use promises was because of performance concerns which have largely been addressed very well by libraries like Bluebird and Q.

As async/await syntax becomes more mainstream, promises will pave the way for code that looks very similar with synchronous code.

Answer 3

While this is still not an actual answer to my question (Q vs async), regarding my problem, I've found Selenium / WebDriverJs to be a viable solution.

driver.get('http://www.google.com');
driver.findElement(webdriver.By.name('q')).sendKeys('webdriver');
driver.findElement(webdriver.By.name('btnG')).click();
driver.wait(function() {
  return driver.getTitle().then(function(title) {
    return title === 'webdriver - Google Search';
  });
}, 1000);

WebDriver uses a queue to execute promises sequentially, which helps immensely with controlling indentation. Its promises are also compatible with Q's.

Creating a sequence of promises is no longer an issue. A simple for-loop will do.

As for stopping early in a sequence, don't do this. Instead of using a sequence, use an asynchronous-while design and branch.