NodeJS generator never reaches a certain line?

Question

I am using co to run a generator function that does some scraping and cleansing of data. However, I never reach a certain part of the code after a loop. Here's what my code looks like:

function*(){
  var url = "mongodb://" + config.host + ":" + config.port + "/" + config.db;
  var db = yield MongoClient.connect( url );
  for( var establishment in [ {"establishment_id": 16} ] ){
    var establishment_type = establishment.id;
    var response = yield getRestaurants( establishment_type );
    var restaurants = JSON.parse( response.body ).restaurants;
    // here we create our restaurants in "saveable" form
    var saveableRestaurants = [];
    for(var i = 0; i < restaurants.length; i++){
        var saveableRestaurant = restaurants[i].restaurant;
        saveableRestaurant._id = restaurants[i].restaurant.R.res_id;
        // Remove unwanted fields
        //... code removed for brevity ...
        // convert cuisine string into its appropriate id form
        var cuisines = saveableRestaurant.cuisines.split(",");
        var ids = [];
        for( var i = 0; i < cuisines.length; i++ ){
            console.log("LOOKING UP ID FOR ", cuisines[i]);
            var match = yield db.collection(CUI).findOne({"cuisine_name":cuisines[i].trim()});
            console.log("ID FOR ", cuisines[i], " IS", match.cuisine_id);
            ids.push( id );
        }
        // I NEVER REACH THIS LINE!!!!
        console.log( "ALL IDS ", ids );
    }
  }
  db.close();
}

I never reach the console statement towards the end of the function

The mongo module is asynchronous. Your db closes before the generator can yield — Sterling Archer, Mar 07 '16 at 15:05
@SterlingArcher So what's the way out? I do not see a `finally( )` or `done( )` in `co` :/ — An SO User, Mar 07 '16 at 15:07
I solved that issue with bluebird and babel. Bluebird for promisification, and babel for ES7 await/async. Here's an [example](https://github.com/RUJodan/SourceUndead/blob/master/routes/lobby.js#L25) of how I did it (with help) — Sterling Archer, Mar 07 '16 at 15:08
with co you would have to yield a promise, then use .then etc like you normally would. it doesn't make asynchronous logic go away. — Kevin B, Mar 07 '16 at 15:08
@KevinB if i move the `db.close( )` to a function of its own in `then( )`, would that solve it? Sorry if my question is basic — An SO User, Mar 07 '16 at 15:10
I don't think so. I've never used **co**, but generators don't wait on asynchronous actions, and co has to abide by that rule as well. — Kevin B, Mar 07 '16 at 15:19
@KevinB I figured it out. I have to wait for all the promises to fulfill so if I collect all the promises in an array and then `yield allPromises`, it'll work. But then how can I get the value from the promise? :/ — An SO User, Mar 07 '16 at 15:20
`gen.next().value.then(doSomethingWithResults)` or, if this is wrapped in **co**, `co(...).then(doSomethingWithResults)` i think (again, never used **co**) — Kevin B, Mar 07 '16 at 15:22
@KevinB: They do wait for yielded promises if you pass them to a "coroutine" runner (like `co` or `Bluebird.coroutine` or `Q.spawn`) — Bergi, Mar 07 '16 at 15:43
@SterlingArcher: `co`/generators + `yield` is just like `async` functions + `await`. There shouldn't be any problem with asynchrony here. — Bergi, Mar 07 '16 at 15:49

score 2 · Answer 1 · answered Mar 07 '16 at 15:47

It's quite likely that there is an asynchronous exception you're not noticing. You should use

function*() {
  var url = "mongodb://" + config.host + ":" + config.port + "/" + config.db;
  var db = yield MongoClient.connect( url );
  try {
    for (var establishment of [ {"establishment_id": 16} ]) {
      var establishment_type = establishment.id;
      var response = yield getRestaurants( establishment_type );
      var restaurants = JSON.parse( response.body ).restaurants;
      // here we create our restaurants in "saveable" form
      var saveableRestaurants = [];
      for (var i = 0; i < restaurants.length; i++) {
        var saveableRestaurant = restaurants[i].restaurant;
        saveableRestaurant._id = restaurants[i].restaurant.R.res_id;
        // Remove unwanted fields
        //... code removed for brevity ...
        // convert cuisine string into its appropriate id form
        var cuisines = saveableRestaurant.cuisines.split(",");
        var ids = [];
        for (var i = 0; i < cuisines.length; i++) {
          console.log("LOOKING UP ID FOR ", cuisines[i]);
          var match = yield db.collection(CUI).findOne({"cuisine_name":cuisines[i].trim()});
          console.log("ID FOR ", cuisines[i], " IS", match.cuisine_id);
          ids.push( id );
        }
        console.log( "ALL IDS ", ids );
      }
    }
  } finally {
    db.close();
  }
}

Also don't forget to put a .catch(e => console.error(e)) on the promise returned by co.

I do have an error handler :) Please have a look at my answer. Also, is there a brain-friendly guide to generators? To me generators are the new regex, given my lack of experience :) — An SO User, Mar 07 '16 at 15:52
You should study promises, synchronous generators and async/await first. With those, `co` is trivial. — Bergi, Mar 07 '16 at 15:56
I understand how Promises work. Generators is the new kid on the block. Any references / guides would be great (beyond the usual MDN, etc.) :) — An SO User, Mar 07 '16 at 15:58
@LittleChild: The first few google hits for `es6 generator` are pretty good: https://davidwalsh.name/es6-generators http://www.2ality.com/2015/03/es6-generators.html — Bergi, Mar 07 '16 at 16:05

score 1 · Accepted Answer · answered Mar 07 '16 at 15:51

1

After scouring through the MongoDB docs, I found the solution:

var matches = db.collection( CUI ).find({"cuisine_name" : { "$in" : cuisines }});
while( yield matches.hasNext() ){
  var match = yield matches.next();
  ids.push( match.cuisine_id );  
}

answered Mar 07 '16 at 15:51

An SO User

24,612
35
133
221

score -1 · Answer 3 · answered Mar 07 '16 at 15:31

-1

You are opening db connection and you are trying to manipulate with db connection , so you can try with async.each and after completing your each functionality you can close the connection, I saw that is the problem lieing down there.

answered Mar 07 '16 at 15:31

Jit

36
2

NodeJS generator never reaches a certain line?

3 Answers3