Chaining Node.js streams with run-time logic

Question

I have been working with Node.js streams for the past 6 months or so, and I have been really happy with them so far. All of the problems I have encountered thus far, I have been able to solve using the standard template of:

A.pipe(B).pipe(C);

However, my current problems requires chaining different stream "pipelines" based on run-time logic. For example, what I'd like to do is something like the following:

var basePipeline = A.pipe(B).pipe(C);
if(flowRate > 0.0) {
    basePipeline.pipe(D).pipe(E).pipe(F);
} else {
    basePipeline.pipe(G).pipe(H).pipe(I);
}

Is the above possible? Or, do I need to create both streams separately in a complete fashion:

if(flowRate > 0.0) {
    A.pipe(B).pipe(C).pipe(D).pipe(E).pipe(F);
} else {
    A.pipe(B).pipe(C).pipe(G).pipe(H).pipe(I);
}

Thanks for the advice!

Because of the way the pipe function returns you should be able to chain them that way. Because the pipe().pipe() works because the second pipe runs off of the return value of the first pipe so if you instead store that return value it will work just the same if you put a .pipe on the end of it. — Binvention, Mar 21 '16 at 15:11

score 1 · Answer 1 · answered Mar 21 '16 at 15:15

What you can do is always pass over all streams, But you keep an array that store if this step have to run or skip. For Example:

if your pipe is like this A -> B -> C -> D -> E -> F -> H

And you have an hash

A:0
B:1
C:0
D:1
...
...

that mean you will run only pipes B and D.

On begin of pipe you check if current pipe is in the hash

// step B
pipe(function(data)){
    if(steps['B'] === 1){
       // do something
    }
}

With this approach, you have fixed pipes, but you can on the fly change the flow.

Chaining Node.js streams with run-time logic

1 Answers1