Returning loop of javascript http requests with promises

Question

I have a function which is looping through an array of urls to be parsed through requests. Because however, I need the for loop to be encompassing the request I don't know how I can return the array. If I have the return of promises outside of the request, I lose the pushed elements since requests is a function. I'm using bluebird for the promises.

Here is what I have so far:

function getSrc (urls, fileList) {
    var path = "#section > a[href]";
    var container = [];
    var i = 0;

    return new Promise(function(fullfill, reject) {
        for (; i < urls.length; i++ ) {         
            request(urls[i], function (err, resp, page) {
                if (!err && resp.statusCode == 200) {
                    var $ = cheerio.load(page); // load the page
                    fileList.push($(path).attr("src"));
                    container.push(fileList);
                    fullfill(container);
                } // end error and status code
                else {reject('error');}
            }); // end request    
        } // end FOR loop
        // fullfill(container); // how can i get this to wait for requests to fill array?
    });
}

score 0 · Answer 1 · answered Aug 11 '15 at 00:45

You cannot use a for-loop in this way because all of your requests will run at the same time and the first to complete will call your fullfill(sic) callback.

The accepted way to accomplish this would be to use Promise.map() to iterate over your urls. Something like:

var request = Promise.promisifyAll(require('request'));

Promise.map(urls,request.get).catch(function(err){
    // oops, one of the requests failed!
  }).then(function(results){
    // do something with the array of all results, one for each url.
  });

Returning loop of javascript http requests with promises

1 Answers1