How to run code using Node.js Fibers

Question

I have a question about Nodejs Fibers(which is absolute new for me) ... I have this tutorial for Nodejs Fibers, http://bjouhier.wordpress.com/2012/03/11/fibers-and-threads-in-node-js-what-for/, and there was an example in here it says

    var fiber = Fiber.current;
    db.connect(function(err, conn) {
    if (err) return fiber.throwInto(err);
       fiber.run(conn);
    });
   // Next line will yield until fiber.throwInto 
   // or fiber.run are called
   var c = Fiber.yield();
   // If fiber.throwInto was called we don't reach this point 
   // because the previous line throws.
   // So we only get here if fiber.run was called and then 
   // c receives the conn value.
   doSomething(c);
   // Problem solved! 

Now based on this Example I created my own version of the code like this,

  var Fiber = require('fibers');

  function sample(callback){
     callback("this callback");
  }

  var fiber = Fiber.current;
  sample(function(string){
     fiber.run(string);
  });
  var string = Fiber.yield();
  console.log(string);

but this gives me an Error of,

/home/ubuntu/Tasks/ServerFilteringV1/test.js:28
    fiber.run(string);
      ^
TypeError: Cannot call method 'run' of undefined

And I have another case which will run a function after 1000 ms with the callback inside (I have done this to test functions with long time executions before a callback),

var Fiber = require('fibers');

function forEach(callback){
   setTimeout(function(){
       callback("this callback");
   },1000);
}


var fiber = Fiber.current;
forEach(function(string){
   fiber.run(string);
});
var string = Fiber.yield();
console.log(string);

This code in here gives me another Error ,

/home/ubuntu/Tasks/ServerFilteringV1/test.js:30
var string = Fiber.yield();
                    ^
Error: yield() called with no fiber running

Well, should the yield() wait after a run() function is executed? Any idea about what is happening in my nodejs code? And thanks in advance ...

Answer 1

Example 1

A fiber is a sort of lightweight thread of execution. Like real threads and processes, a fiber must be given a block of code to execute upon run. The code as you took it from bjouhier does not work as is. It was intended to run inside a fiber, like so:

var f = Fiber(function() {
    var fiber = Fiber.current;

    sample(function(str) {
        fiber.run(string);
    });

    var str = Fiber.yield();
    console.log(str);
});

f.run();

Calling run on the fiber, well, runs the fiber code, which was given as callback to Fiber. The above code, however, will also give an error (stating the fiber is already running). One might easily see why when analysing the order of execution.

1. Set variable f as a fiber.
2. Run the fiber:
   1. Set variable fiber pointing to the current running fiber.
   2. Call function sample.
   3. Call the callback.
   4. Call fiber.run, which gives the error as the current fiber is already running.

The structure of this code is correct, but it assumes sample is some asynchronous function that does not immediately call the callback. Let's swap out your sample function by this one:

function sample(callback) {
    setTimeout(function() {
        callback("this callback");
    }, 500);
}

Now, the above code won't emit an error, as sample immediately returns. The order of execution inside the fiber is:

1. Set fiber pointing to the current running fiber.
2. Call sample, which returns without calling the callback (yet).
3. Call `Fiber.yield(), which 'pauses' the current fiber.
4. After 500 ms, call the callback.
5. Call fiber.run() passing 'this callback', which resumes the fiber.
6. Fiber.yield returns, Set str to 'this callback'.
7. Log the string to console.

Observe that step 4 is done outside the execution of the fiber.

Example 2

Whereas in the first example there was no running fiber (and therefore fiber was undefined), in the second example the error is thrown for the same reason. Again, the code needs to run inside a fiber.

---

The function of yield and run

A fiber must cooperatively give control to another fiber (or the main line of execution). Compare that to the preemptive nature of threads and processes. The giving up control is what is meant by 'yielding control', and in this case is done by Fiber.yield().

To continue execution (directly after the point of where the fiber yielded), one must call run() on the fiber.

The mechanism to pass values in and out of the fiber is through the interplay of yield and run:

• An argument given to run (which is outside the fiber), is returned by yield (inside the fiber).
• An argument given to yield (inside the fiber), is returned by run (outside the fiber).

For an example, look at the incremental generator on the github repository of node-fibers. Additionally, observe that our Example 1, the callback given to sample is essentially ran outside the fiber, as it is ran on the next tick (viz. the async nature of setTimeout).

Answer 2

As explained by Andrew, and as hinted in my blog post (see the sentence that follows the example), you have to create a Fiber and run it with run() to be able to call Fiber.yield.

The benefit of fibers is not obvious when you have a single async call to run, but consider the case where you have a function f1 that calls f2 that calls f3. If f3 calls a low-level async function with a callback and if you do not use fibers, you have to turn f3 into an async function with a callback, and then by contagion you also have to turn f2 and f1 into async functions. With fibers, you can keep f1, f2 and f3 as normal functions (without callback). You will need some Fiber.yield() magic inside f3 and you will also need to call f1 from inside a Fiber but you don't need to worry about callbacks in the bodies of f1 and f2.

So fibers really shine when you have multiple layers of code or complex control flow between your high level functions and the low level async functions that they call.

Also, Marcel, who wrote fibers, recommends that you don't use Fiber.yield() directly in your code but that you use his futures library instead. It's interesting to play with Fiber.yield to understand what fibers are made of but I encourage you to use the futures library for a real project. It will also help you parallelize your code.