Transform JS objects to JSON using transform stream

Question

Note there are many transform streams that do this:

JSON -> JS

but I am looking to create a Node.js transform stream that can do:

JS -> JSON

I have a readable stream:

const readable = getReadableStream({objectMode:true});

the readable stream outputs Objects, not strings.

I need to create a transform stream which can filter some of those objects and convert the objects to JSON, like so:

const t = new Transform({
  objectMode: true,
  transform(chunk, encoding, cb) {
    if(chunk && chunk.marker === true){
       this.push(JSON.stringify(chunk));
     }
    cb();
  },
  flush(cb) {
    cb();
  }
});

however, for some reason my transform stream cannot accepts objects to the transform method, only string and buffer, what can I do?

I tried adding this two options:

  const t = new Transform({
      objectMode: true,
      readableObjectMode: true,  // added this
      writableObjectMode: true,  // added this too
      transform(chunk, encoding, cb) {
        this.push(chunk);
        cb();
      },
      flush(cb) {
        cb();
      }
    });

unfortunately my transform stream still cannot accept objects, only string/buffer.

score 8 · Accepted Answer · answered Mar 16 '18 at 21:24

You need to use only writableObjectMode: true on transform stream.

Documentation

options <Object> Passed to both Writable and Readable constructors. Also has the following fields:
    readableObjectMode <boolean> Defaults to false. Sets objectMode for readable side of the stream. Has no effect if objectMode is true.
    writableObjectMode <boolean> Defaults to false. Sets objectMode for writable side of the stream. Has no effect if objectMode is true.

You want writeable part of your transform stream to accept objects because objects are written to it. While strings will be read from it.

Check out this minimal working example:

const { Readable, Writable, Transform } = require('stream');

let counter = 0;

const input = new Readable({
  objectMode: true,
  read(size) {
    setInterval( () => {
      this.push({c: counter++});  
    }, 1000);  
  }  
});

const output = new Writable({
  write(chunk, encoding, callback) {
    console.log('writing chunk: ', chunk.toString());
    callback();  
  }  
});

const transform = new Transform({
  writableObjectMode: true,
  transform(chunk, encoding, callback) {
    this.push(JSON.stringify(chunk));
    callback();  
  }  
});

input.pipe(transform);
transform.pipe(output);

Transform JS objects to JSON using transform stream

1 Answers1