Managing asynchronous Callbacks in Meteor

Question

I'm using Meteor 1.* and Iron Router 1.*.

I'm using Node.js calls on the server side in Meteor, outside of a Meteor-method -- specifically inside a server side Iron Router route.

A portion of the code inside of the route looks similar to the following so far:

  fs.mkdir(filebox, function (e) {
    if(!e || e.code === 'EEXIST') {
      fs.writeFile(file1, function (err) {
        if (err) throw err;

        fs.writeFile(file2, function (err) {
          if (err) throw err;

          fs.writeFile(file.3, function (err) {
            if (err) throw err;

            ExternalLibrary.do_something_(file1, function (err, buffer) {
              if (err) throw err;

              ExternalLibrary.do_something_(file2, function (err, buffer) {
                if (err) throw err;

                ExternalLibrary.do_something_(file3, function (err, buffer) {
                  if (err) throw err;

                  some_object = { first: file1, second: file2 }

                  ExternalLibrary.do_something_else_(some_object, function (err, buffer) {
                    if (err) throw err;

                    fs.readFile(file1, function (err, data) {
                      if (err) throw err;

                      res.write(data);
                      res.end();
                    });
                  });
                });
              });
            });
          });
        });
      });
    } else {
      console.log(e);
    }
  });

My problem is, I need to add even more calls to fs.write and the ExternalLibrary and further make these calls conditional.

It looks like I'm entering Callback hell.

On Callbacks

I know that Meteor uses Coroutines (or fibers, or continuations), but I don't know anything about how this works. And that within a Meteor-method we have the option of using Meteor.wrapAsync.

I've read some on Node.js Promises and Generators. And specifically I'm trying out the frozeman/q-meteor library.

Question

What is the best way to 'flatten' this tree and save myself from Callback hell? I want a solution that will allow for conditional method calls too. For example, I'm going to eventually need to add something like the following to the code example above:

if (some_condition === true) {
  // call this method or node function
  fs.writeFile(file4, function (err) {
    fs.writeFile(file5, function (err) {
      // do something
    }
  }
}
else {
  // call this method or node function
  fs.writeFile(file6, function (err) {
    fs.writeFile(file7, function (err) {
      // do something
    }
  }
}

Aaron · Answer 1 · 2015-03-09T20:10:04.053

I ended up using Promises for the solution. I used the excellent Q library by Kris kriskowal/q. Specifically the Meteor wrapper for this is frozeman/q-meteor.

The following is the code:

makeDirectory(filebox)
  .then(function () {
    console.log('in write file function');
    return writeFile(file1, data);
  })
  .then(function () {
    console.log('in write file function');
    return writeFile(file2, data);
  })
  .then(function () {
    console.log('in write file function');
    return writeFile(file3, data);
  })
  .then(function () {
    console.log('in external lib function');
    return ExternalLibrary._do_something(file1, data1, output1);
  })
  .then(function () {
    console.log('in external lib function');
    return ExternalLibrary._do_something(file2, data2, output2);
  })
  .then(function () {
    console.log('in external lib function');
    return ExternalLibrary._do_something(file3, data3, output3);
  })
  .then(function () {
    console.log('in concat function');
    return ExternalLibrary.cat(fileToCat, outputCat);
  })
  .then(function () {
    console.log('in read file function');
    return readFile(outputCat);
  })
  .then(function (result) {
    console.log('in the response function');
    res.write(result);
    res.end();
  })
  .catch(function (error) {
    console.log('darn error: ', error);
  })
  .done(function (result) {
    console.log('done');
  });

Both the External Library and function calls in each .then return a deferred promise, the output of which can be used in the preceding .then function call.

A couple of links that helped me get started:

Any reason why you don't do all of this in parallel? You're never using any of the previous results. — Bergi, Mar 11 '15 at 17:47
The calls to the external library are async as are the calls to the fs.write functions. Each write function needs to be in sequence because the file created is used in the external libraries. The example I made is not complete, because as you point out I'm not using the Promised callback values... my actual code does use the deferred.resolve(variable_name) and the function in the .then call uses this variable. I might up this example with one more complete. — Aaron, Mar 11 '15 at 18:06
Yes, please post complete code that shows the dependencies :-) — Bergi, Mar 11 '15 at 18:08

Managing asynchronous Callbacks in Meteor

On Callbacks

Question

1 Answers1