yielding from an iterator callback used inside a generator

Question

Has anyone tried to get Underscore JS or lodash (or any ES5 standard functions for that matter) working with generators?

If we have an array var myArray = [1,2,3,4,6]; We want to forEach over it.

In a non generator case you would simply

myArray.forEach(function(k) {
  console.log(k);
});

However, when you can't yield inside a non generator function, so if inside this loop we had to do some async work, you would need to do the following.

var foreach = function* (arr, fn) {
  var i;
  for (i = 0; i < arr.length; i++) {
    yield * fn(arr[i], i);
  }
};

yield* foreach(myArray, function* (k) {
  var a = yield fs.readFile();
});

Which kind of sucks.

Anyone know of a way to get anonymous functions working with generators? We kind of lose the entire lodash library because of this.

Note: I'm using Traceur to compile my code into ES6 with generators turned on.
Note: I'm not using co(). I'm using a custom generator function seen below

var run = function(generatorFunction) {
  var generatorItr = generatorFunction(resume);
  function resume(callbackValue) {
    generatorItr.next(callbackValue);
  }
  generatorItr.next();
};

Maybe it's just me, but I don't understand what exactly the problem is. It sounds more like the problem is using e.g. `forEach` with generators. — Felix Kling, Jul 26 '14 at 04:51
Well yes, but that's not the real* problem. The problem is using yield inside of a non generator function. Which ForEach will use 90% of the time. Not to mention _.find(), _.filter(), Array.reduce(), Array.forEach(), Array.map(). All of them are useless if you need to yield anything inside. — Sean Clark, Jul 26 '14 at 17:07
In case of `forEach`, you can simply use `for (var e of arr) { yield doSomethingWith(e); }` or a normal `for` loop. For other methods such as `filter` or `reduce`, I don't see how using a generator would even be useful. The `filter` callback must return a boolean. How exactly would using a generator make sense here? — Felix Kling, Jul 26 '14 at 18:25
Well, any answer I can think of would be inefficient code. Doing work inside a loop. But anyway, if you have a list of mp3 urls and you need to filter that list down to ones that actually exist on the filesystem. You would normally do a filter on your list, check the FS on each iteration, and Promise.all() when they all are done. Using generators we can't use filter. We have to loop and store a 2nd array of the results. — Sean Clark, Jul 26 '14 at 18:34
@SeanClark: Notice that `Promise.all` would have started the fs queries in parallel, while the generator solution you seem to be looking for would be sequential. — Bergi, Jul 26 '14 at 19:29
That is true, however you can yield afterwards, making them parallel again. You can push the fs queries onto an array and yield the array. Similar to promises, but not needing create promises and resolve them all and then in another block Promise.all — Sean Clark, Jul 28 '14 at 15:44

score 1 · Answer 1 · answered Aug 04 '14 at 19:52

If I'm understanding your problem correctly, it's essentially that you're trying to do something (iterate until a good stopping point is found) in an asynchronous way, in a language (JS) which is really designed around synchronicity. In other words, while you could normally do:

_([1,2,3]).any(function(x) {
    var shouldWeStopLooping = x % 2 == 0;
    return shouldWeStopLogging;
});

you instead want to make the "should we stop looping" code break from normal execution, and then come back, which isn't possible with traditional JS (yield is relatively new to the language) and thus isn't possible in Underscore/Lodash:

_([1,2,3]).any(function(x) {
    var shouldWeStopLooping = $.ajax(...); // Doesn't work; code keeps going
    return shouldWeStopLogging;
});

There are two approaches you could take, neither of which are ideal.

As mentioned in the comments, one approach would be to do all your "deferred" work first, then iterate:

var workInProgress = _([1,2,3]).map(someAjaxOperation);
$.when.apply(workInProgress).done(doSomethingBasedOnAjaxResults);

But (as also noted in the comments) that isn't quite the same, as you wind up doing the AJAX work on all of the elements of your array (vs. a true generator which would only iterate through as many as needed to find a "winner").

Another approach would be to eliminate the asynchronicity. jQuery allows you to pass async: false to an AJAX request, which "solves" the problem by letting you use Underscore/Lodash/whatever ... but it also locks your user's browser up for as long as it takes to do the AJAX work, which probably isn't what you want.

Unfortunately if you want to use a library like Underscore/Lodash those are the only options I can see. Your only other option would be to write your own Underscore/Lodash mix-in, which really isn't that hard. I'd recommend doing this, as it would allow you still leverage all the other great functions in those libraries while still iterating in a consistent way.

Not exactly. I don't need to stop the loop when x is found. I just am wanting to use lodash loop features. Like filter and reduce. Or ES6 reduce for that matter. There are cases where I want things to happen in a loop that can be async, but i want a nicer way to write that code. That's the point of this whole thing, write nicer code. — Sean Clark, Aug 06 '14 at 04:27
Also, I agree your jQuery async: false, is a solution on the client, but not really the case here either. I want things to be async because this is NodeJS. I just want the code I write to stay out of callback hell. I wanted a good solution to use loops, but right now it looks like yield can't be use in a closure situation, it has to be within a gen function. ES7 seems to have async functions - which may be the answer. — Sean Clark, Aug 06 '14 at 04:29

yielding from an iterator callback used inside a generator

1 Answers1