what's differnce between push() and callback in nodejs Transform

Question

i'm implementing a Transform stream which is piped by mongoose cursor (which works like readable stream, (or maybe it's real readable stream))

const myTransform = new Transform({
  readableObjectMode: true,
  writableObjectMode: true,
  transform(chunk: Document, encoding, callback) {
    this.push(chunk?.toObject() ?? null); // <- transforms data
    callback(); // <- calls callback
  },
});

MyMongooseModelWithHugeData.find()
  .cursor()
  .pipe(myTransform)
  .pipe(someWriteStream)

Yup. My current code works fine.

But I found out the callback inside transform implementation, receives second parameter (which looks like classic nodejs async callback style obviously).

So, I changed my code like below, and found out that it also works fine..

const myTransform = new Transform({
  readableObjectMode: true,
  writableObjectMode: true,
  transform(chunk: Document, encoding, callback) {
    callback(null, chunk?.toObject() ?? null); // <- transforms data and calls callback.
  },
});

And I felt somthing awkward. I've search some blogs about creating Transform stream. And all articles that I found was telling same that I have to use this.push() and call callback(). But none of them did even mention about the second parameter of callback().

So, my question is:

Is there's difference between this.push() and callback() when passing transformed value?
Why many people recommend using this.push instead of callback ?

They are semantically identical when forwarding a single chunk. However, `push()` may be called multiple times, while `callback()` must be called only once. I tend to use `callback(null, obj)` when only forwarding a single chunk and, otherwise, repeated `push(obj)` + `callback()` when relaying multiple chunks per handler execution. — jorgenkg, Apr 21 '22 at 19:21

score 5 · Accepted Answer · answered May 01 '22 at 10:21

from nodejs documantation

https://github.com/nodejs/node/blob/master/doc/api/stream.md#transform_transformchunk-encoding-callback

It is possible that no output is generated from any given chunk of input data.

The callback function must be called only when the current chunk is completely consumed. The first argument passed to the callback must be an Error object if an error occurred while processing the input or null otherwise. If a second argument is passed to the callback, it will be forwarded on to the transform.push() method.

In other words, the following are equivalent:

transform.prototype._transform = function(data, encoding, callback) {
  this.push(data);
  callback();
};

transform.prototype._transform = function(data, encoding, callback) {
  callback(null, data);
};

what's differnce between push() and callback in nodejs Transform

1 Answers1