How do you carry mutating data into callbacks within loops?

Question

I constantly run into problems with this pattern with callbacks inside loops:

while(input.notEnd()) {
    input.next();
    checkInput(input, (success) => {
        if (success) {
            console.log(`Input ${input.combo} works!`);
        }
    });
}

The goal here is to check every possible value of input, and display the ones that pass an asynchronous test after confirmed. Assume the checkInput function performs this test, returning a boolean pass/fail, and is part of an external library and can't be modified.

Let's say input cycles through all combinations of a multi-code electronic jewelry safe, with .next incrementing the combination, .combo reading out the current combination, and checkInput asynchronously checking if the combination is correct. The correct combinations are 05-30-96, 18-22-09, 59-62-53, 68-82-01 are 85-55-85. What you'd expect to see as output is something like this:

Input 05-30-96 works!
Input 18-22-09 works!
Input 59-62-53 works!
Input 68-82-01 works!
Input 85-55-85 works!

Instead, because by the time the callback is called, input has already advanced an indeterminate amount of times, and the loop has likely already terminated, you're likely to see something like the following:

Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!

If the loop has terminated, at least it will be obvious something is wrong. If the checkInput function is particularly fast, or the loop particularly slow, you might get random outputs depending on where input happens to be at the moment the callback checks it.

This is a ridiculously difficult bug to track down if you find your output is completely random, and the hint for me tends to be that you always get the expected number of outputs, they're just wrong.

This is usually when I make up some convoluted solution to try to preserve or pass along the inputs, which works if there is a small number of them, but really doesn't when you have billions of inputs, of which a very small number are successful (hint, hint, combination locks are actually a great example here).

Is there a general purpose solution here, to pass the values into the callback as they were when the function with the callback first evaluated them?

Answer 1

If you want to iterate one async operation at a time, you cannot use a while loop. Asynchronous operations in Javascript are NOT blocking. So, what your while loop does is run through the entire loop calling checkInput() on every value and then, at some future time, each of the callbacks get called. They may not even get called in the desired order.

So, you have two options here depending upon how you want it to work.

First, you could use a different kind of loop that only advances to the next iteration of the loop when the async operation completes.

Or, second, you could run them all in a parallel like you were doing and capture the state of your object uniquely for each callback.

I'm assuming that what you probably want to do is to sequence your async operations (first option).

Sequencing async operations

Here's how you could do that (works in either ES5 or ES6):

function next() {
    if (input.notEnd()) {
        input.next();
        checkInput(input, success => {
            if (success) {
                // because we are still on the current iteration of the loop
                // the value of input is still valid
                console.log(`Input ${input.combo} works!`);
            }
            // do next iteration
            next();
        });
    }
}
next();

Run in parallel, save relevant properties in local scope in ES6

If you wanted to run them all in parallel like your original code was doing, but still be able to reference the right input.combo property in the callback, then you'd have to save that property in a closure (2nd option above) which let makes fairly easy because it is separately block scoped for each iteration of your while loop and thus retains its value for when the callback runs and is not overwritten by other iterations of the loop (requires ES6 support for let):

while(input.notEnd()) {
    input.next();
    // let makes a block scoped variable that will be separate for each
    // iteration of the loop
    let combo = input.combo;
    checkInput(input, (success) => {
        if (success) {
            console.log(`Input ${combo} works!`);
        }
    });
}

Run in parallel, save relevant properties in local scope in ES5

In ES5, you could introduce a function scope to solve the same problem that let does in ES6 (make a new scope for each iteration of the loop):

while(input.notEnd()) {
    input.next();
    // create function scope to save value separately for each
    // iteration of the loop
    (function() {
        var combo = input.combo;
        checkInput(input, (success) => {
            if (success) {
                console.log(`Input ${combo} works!`);
            }
        });
    })();
}

Answer 2

You could use the new feature async await for asynchronous calls, this would let you wait for the checkInput method to finish when inside the loop.

You can read more about async await here

I believe the snippet below achieves what you are after, I created a MockInput function that should mock the behaviour of your input. Note the Async and await keywords in the doAsyncThing method and keep an eye on the console when running it.

Hope this clarifies things.

function MockInput() {
  this.currentIndex = 0;
  this.list = ["05-30-96", "18-22-09", "59-62-53", "68-82-0", "85-55-85"];
  
  this.notEnd = function(){
    return this.currentIndex <= 4;
  };
  
  this.next = function(){
      this.currentIndex++;
  };
  
  this.combo = function(){
      return this.list[this.currentIndex];
  }
}

function checkInput(input){
  return new Promise(resolve => {
      setTimeout(()=> {
        var isValid = input.currentIndex % 2 > 0; // 'random' true or false
        resolve( `Input ${input.currentIndex} - ${input.combo()} ${isValid ? 'works!' : 'did not work'}`);
      }, 1000);
  });
}

async function doAsyncThing(){
  var input = new MockInput();
  
  while(input.notEnd()) {
      var result = await checkInput(input);
      console.log(result);
      input.next();
  }
  
  console.log('Done!');
}

doAsyncThing();