Q Promise chain and NodeJS callbacks

Question

I think this is a really stupid question but I'm having a hard time wrapping my head around promises.

I'm using Q (for nodejs) to sync up a couple of async functions. This works like a charm.

    var first = function () {
        var d = Q.defer();
        fs.readdir(path,function(err,files){
            if(err) console.log(err);
            d.resolve(files);
        });
        return d.promise;
    };

    var second = function (files) {
        var list = new Array;
        files.forEach(function(value, index){
            var d = Q.defer();
            console.log('looking for item in db', value);
            db.query(
                'SELECT * FROM test WHERE local_name =? ', [value],{
                    local_name      : String,

                },
                function(rows) {
                    if (typeof rows !== 'undefined' && rows.length > 0){
                        console.log('found item!', rows[0].local_name);
                        d.resolve(rows[0]);
                    } else {
                        var itemRequest = value;
                        getItemData(itemRequest);
                    }
                }
            );
            list.push(d.promise);
        });
        return Q.all(list);
    };

    first()
    .then(second)
    .done(function(list){
        res.send(list);
    });

The problem I have is with this little function:

  getItemData(itemRequest) 

This function is filled with several of callbacks. The promise chain runs through the function just fine but ignores all the callbacks I use ( eg several XHR calls I make in the function).

A simplified version of the function looks like this (just to give you an idea):

    function getItemData(itemRequest){
        helper.xhrCall("call", function(response) {
            var requestResponse = JSON.parse(response)
            , requestInitialDetails = requestResponse.results[0];

            downloadCache(requestInitialDetails,function(image) {

                    image = localImageDir+requestInitialDetails.image;

                    helper.xhrCall("call2", function(response) {

                        writeData(item,image,type, function(){
                            loadData(item);
                        });
                    });
                } else {
                    writeData(item,image,type, function(){
                        loadData(item);
                    });
                }
            });
        });

The xhr function I use looks like this:

  xhrCall: function (url,callback) {
    var request = require("request")
    , colors = require('colors');
    request({
        url: url,
        headers: {"Accept": "application/json"},
        method: "GET"
    }, function (error, response, body) {
        if(!error){
            callback(body);
        }else{
           console.log('Helper: XHR Error',error .red); 
        }
    });
  }

So my questions:

• Can I leave the function unaltered and use the callbacks that are in place ánd the promise chain?
• Or do I have to rewrite the function to use promises for the XHR?
• And if so, How can I best write my promise chain? Should I reject the initial promise in the forEach?

Again, sorry if this is a really stupid question but I don't know what the right course of action is here.

Thanks!

[EDIT] Q.nfcall, I don't get it

So I've been looking into Q.nfcall which allows me to use node callbacks. Bu I just don't understand exacly how this works. Could someone give a simple example how I would go about using it for a function with several async xhr calls?

I tried this but as you can see I don't really understand what I'm doing:

    var second = Q.nfcall(second);

    function second (files) {

[EDIT 2]

This is the final funcction in my getitemdata function callback chain. This function basically does the same as the function 'second' but I push the result directly and then return the promise. This works as stated, but without all the additional callback data, because it does not wait for the callbacks to return with any data.

  function loadData(item) {
        var d = Q.defer();
        db.query(
            'SELECT * FROM test WHERE local_name =? ', [item],{
                local_name      : String,

            },
            function(rows) {
                if (typeof rows !== 'undefined' && rows.length > 0){
                    list.push(d.promise);
                } 
            }
        );

    });
    return Q.all(list);
};

Answer 1

Your answer is not really clear after your second edit.

First, on your orignal question, your getItemData has no influence on the promise chain.
You could change you the function's call signature and pass your deferred promise like so.

getItemData(itemRequest, d) 

and pass this deferred promises all the way to your xhrCall and resolve there.

I would re-write your whole implementation and make sure all your functions return promises instead.

Many consider deferred promises as an anti-pattern. So I use use the Promise API defined in harmony (the next javascript)

After said that, I would re-implement your original code like so (I've not tested)

var Promise = Promise || require('es6-promise').Promise // a polyfill
;

function errHandler (err){
  throw err
}

function makeQuery () {
  var queryStr = 'SELECT * FROM test WHERE local_name =? '
  , queryOpt = {local_name: String}
  ;
  console.log('looking for item in db', value)

  return new Promise(function(resolve, reject){
    db.query(queryStr, [value], queryOpt, function(rows) {
      if (typeof rows !== 'undefined' && rows.length > 0){
        console.log('found item!', rows[0].local_name);
        resolve(rows[0]);
      } else {
        // note that it returns a promise now.
        getItemData(value).then(resolve).catch(errHandler)
      }
    })
  })
}

function first () {
  return new Promise(function(resolve, reject){
    fs.readdir(path, function(err, files){
      if (err) return reject(err)
      resolve(files)
    })
  })
}

function second (files) {
  return Promise.all(files.map(function(value){
    return makeQuery(value)
  });
}

first()
.then(second)
.then(res.send)
.catch(errHandler)

Note that there is no done method on the Promise API.

One down side of the new Promise API is error handling. Take a look at bluebird.
It is a robust promise library which is compatible with the new promise API and has many of the Q helper functions.

Answer 2

As far as I can tell, you need to return a promise from getItemData. Use Q.defer() as you do in second(), and resolve it when the callbacks complete with the data. You can then push that into list.

To save code, you can use Q.nfcall to immediately call a node-style-callback function, and return a promise instead. See the example in the API docs: https://github.com/kriskowal/q/wiki/API-Reference#qnfcallfunc-args