Stack Overflow
Stack Overflow

Questions tagged [transform-stream]

Refers to NodeJS Transform Stream functions, a form of `duplex` stream function that mutates the streamed data.

From the NodeJS Stream docs

Transform streams are Duplex streams where the output is in some way related to the input. Like all Duplex streams, Transform streams implement both the Readable and Writable interfaces.

15 questions
3
votes
1 answer

How do I get a NodeJS Transform stream to emit multiple records for each chunk of input?

I am writing a Transform stream that takes an incoming stream of XML and emits a subset of that as JS objects. some name needs…
Dave Sag
  • 13,266
  • 14
  • 86
  • 134
1
vote
0 answers

How to Convert a JSON Stream to NDJSON easily

How can I convert a fetch response stream where the document being downloaded is JSON to a NDJSON stream? For example this code: const response = await fetch(url); ndjsonStreamer = ndjsonStream(response.body).getReader(); works well for ndjson…
Victor Ekpo
  • 11
  • 4
1
vote
0 answers

Throwing an error from a Node.js Transform stream

I need to throw an error in a Transform stream. Normally, I'd do this with the callback function on _transform(). I can't in my situation because I need to throw the error even if no data is currently flowing through my stream. That is, if no data…
Brad
  • 159,648
  • 54
  • 349
  • 530
1
vote
1 answer

Shopify Server not allowing complete file download

I have coded a page using webpack and integrated it into a shopify store. The site is working fine on my local server, but not on the shopify servers. The problem is due to this part:  return fetch(filePath) // Retrieve its body as…
Minichua
  • 185
  • 14
1
vote
1 answer

Huffman's algorithm with stream in Node JS

I have implemented Huffman's algorithm in Node JS and it looks like this: huffman.encode(inputFilename, outputFilename) huffman.decode(inputFilename, outputFilename) But I want to implement it like…
Sergei Martianov
  • 301
  • 1
  • 9
1
vote
2 answers

Discard chunk in NodeJS Transform stream and read the next one

How do I discard a chunk in a NodeJS transform stream and read the next one from the previous stream? stream.on('data',function(data){}).pipe(rulesCheck).pipe(step1).pipe(step2).pipe(step3).pipe(insertDataIntoDB); I would like to discard the chunk…
user2405589
  • 881
  • 2
  • 14
  • 32
1
vote
2 answers

how to use node transform to transform one object into another

I'm new to Node streams and trying to figure out why things aren't working as expected. I'm just practicing and want to do a silly transform. interface MyTransformStreamOptions { [key: string]: any } class MyTransformStream extends Transform { …
lostdorje
  • 6,150
  • 9
  • 44
  • 86
0
votes
1 answer

How to filter an array of objects stored as a .json.gz on S3 using Streams chunks in nodejs

Let's say the json array stored in s3 is person details such as: [{"name":'A', "lastName":'A', age: 18}, {"name":'B', "lastName":'B', age: 20}, ...] This file could be extremely large and I would like to optimize memory usage and use Streams to…
Ananthalakshmi Sankar
  • 735
  • 5
  • 7
0
votes
0 answers

Spawned ffmpeg process in nodejs Transform stream with flow control doesn't process input stream

I have implemented a node.js Transform stream class that spawns ffmpeg and streams out the transformed stream at a controlled realtime rate. Below is my _transform() method. this.rcvd += chunk.length console.log('received bytes %d', this.rcvd) const…
user1832894
  • 25
  • 5
0
votes
0 answers

How can backpressure be applied programmatically with a javascript TransformStream?

According to the spec for TransformStreams at https://streams.spec.whatwg.org/#pipe-chains we are told as follows: Once a pipe chain is constructed, it will propagate signals regarding how fast chunks should flow through it. If any step in the…
Graham Leggett
  • 911
  • 7
  • 20
0
votes
1 answer

Two confusions about transform stream part of node.js documentation around pre-ES6 style

Here is the doc I am confused with. When using pre-ES6 style constructors const { Transform } = require('stream'); const util = require('util'); function MyTransform(options) { if (!(this instanceof MyTransform)) return new…
krave
  • 1,629
  • 4
  • 17
  • 36
0
votes
1 answer

NodeJS net.socket Pipe by condition/filter

I'm new to NodeJS. The existing net.socket pipe need to filter by condition to not to connect to "con2", my existing code as follow. I found Transform and PipeLine methods and, so far I tried, the sample code not working for my scenario yet. The…
Thet Thet
  • 13
  • 5
0
votes
1 answer

Problem during testing transform-streams (Node.js) with Benchmark.js

I'm trying to benchmark a NodeJS code but I get the following error: events.js:167 throw er; // Unhandled 'error' event ^ Error [ERR_STREAM_WRITE_AFTER_END]: write after end at writeAfterEnd (_stream_writable.js:243:12) at…
Andrea Simone Costa
  • 1,164
  • 6
  • 16
0
votes
2 answers

How to write an async function that resolves when `data` event emitter fires

I am using node-serialport to communicate with a piece of hardware. It just writes a command and receives a response. https://serialport.io/docs/en/api-parsers-overview The following code works: const port = new SerialPort(path); const parser =…
Nick Young
  • 885
  • 1
  • 10
  • 21
0
votes
1 answer

wrapping nodejs stream in JSON object

I have a readable stream, something like this: const algorithm = 'aes-256-ctr'; stream = file.stream .pipe(crypto.createCipher(algorithm, encryptionKey)) .pipe(outStream); Encryption works as expected on the entire file. I need to wrap the…
alexg
  • 902
  • 11
  • 37