Run an asynchronous call of Cheerios with a Promise

Question

I have a promise call that is supposed to do the following:

1) Get a link from completedLinks

2) Use cheerio to load the html into the $, like jQuery.

3) get the title, and other details from the page

4) save those details to the object siteObj

JS:

completedLinks = ["http://www.example.com/1","http://www.example.com/2","http://www.example.com/3"...]

   function fetchSiteDetails(arr) {
      return arr.reduce(function(promise, link) {
        console.log(link);
        var siteObj = {};

        var $ = cheerio.load(link);
        var titles = $("title").text().trim();  
        var html = $("#content").html().trim();
        var mainContent = $(".left-side").html().trim();
        var allLinks = $('a');

        siteObj.title = titles;
        siteObj.html = html;
        siteObj.mainContent = mainContent;
        siteObj.subPages = [];

       allLinks.each(function(i, elem) {
         siteObj.subPages.push($(this).attr('href'));
       });

        return promise;
      }, Promise.resolve());
    }

    fetchSiteDetails(completedLinks).then(function() {
        console.log('all done');
    });

Right now it is not completing each task before it starts the next, and finishes before it's done anything.

you're returning the one resolved promise in each iteration of the reduce function - which part of the reduce code is at all asynchronous, I can't tell from the code ... what is `completedLinks` an array of? — Jaromanda X, Jan 28 '16 at 23:14

score 1 · Answer 1 · answered Jan 28 '16 at 23:33

As I can't see what is asynchronous in your code, here's a generic pattern to run asynchronous operations in series using promises and array.reduce

return arr.reduce(function(promise, link) {
    return promise.then(function(resultOfPreviousPromise) {
        // your code goes here
        // return a promise that resolves when the asynchronous function completes
    });
}, Promise.resolve());

What will interest you is the bit where that says "your code goes here" - which is where you should put your code. The return value needs to be a promise that is resolved when the asynchronous operation completes

Run an asynchronous call of Cheerios with a Promise

1 Answers1