Stack Overflow
Stack Overflow

Questions tagged [node.js-stream]

A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.

There are many stream objects provided by Node.js. For instance, a request to an HTTP server and process.stdout are both stream instances.

Streams can be readable, writable, or both. All streams are instances of EventEmitter.

The stream module can be accessed using:

const stream = require('stream');

There are four fundamental stream types within Node.js:

  • Readable - streams from which data can be read (for example fs.createReadStream()).
  • Writable - streams to which data can be written (for example fs.createWriteStream()).
  • Duplex - streams that are both Readable and Writable (for example net.Socket).
  • Transform - Duplex streams that can modify or transform the data as it is written and read (for example zlib.createDeflate()).
205 questions
3
votes
1 answer

Proper use of sockets and/or Celluloid::IO

I have a Pinoccio microcontroller (absolutely awesome, try them). The microcontroller opens a socket to it's server. I am writing that TCP Socket server in a Ruby application, in which I will use Celluloid::IO. As my guide, I am following this…
roder
  • 566
  • 6
  • 13
3
votes
0 answers

Memory consideration using buffer and buffer.toString() in node.js streams

I am performing readstream.pipe(transformstream).pipe(writeStream). The application is read xml from a file system using readstream --> do some manipulation based on different tags using transform stream --> write it to another file system. During…
user3124360
  • 353
  • 4
  • 10
3
votes
1 answer

node.js http.IncomingMessage does not fire 'close' event

When does the http.IncomingMessage fire its 'close' event? According to the documentation it should occur when the underlaying connection was closed. However, it is never called for the following example code (I made sure it is not caused by…
user1824468
2
votes
0 answers

Node.js readable maximize throughput/performance for compute intense readable - Writable doesn't pull data fast enough

General setup I developed an application using AWS Lambda node.js 14. I use a custom Readable implementation FrameCreationStream that uses node-canvas to draw images, svgs and more on a canvas. This result is then extracted as a raw image buffer in…
flohall
  • 967
  • 10
  • 19
2
votes
0 answers

How to handle 'end'/'close' events in Transform Node.js stream?

Attaching a transform stream via .pipe doesn't seem to handle all the events automatically. This example: const stream = require('stream'); const readable1 = stream.Readable.from([1, 2, 3]) .on('data', (data) => console.log(data)) .on('end', ()…
Valentyn Derkach
  • 401
  • 3
  • 8
2
votes
3 answers

Propagating Node.js stream error events to async await style code

I use readable and transform streams which I later consume using for await. I cannot find a way to process callee's stream errors so they can be caught in the caller function. For example if transform throws, it results in uncaught error. If I add…
krl
  • 5,087
  • 4
  • 36
  • 53
2
votes
1 answer

NodeJS Writable streams writev alternating between 1 and highWaterMark chunks

So I have a stream that generates data and one that writes them to the database. Writing to the databse is slow. I use the writev function to write a batch of 3000 chunk at once. const generator = new DataGenerator(); // extends Readable const…
SmallhillCZ
  • 152
  • 10
2
votes
2 answers

what is difference Between Package and Module in node.js

By using command npm install upper-case i can download upper-case package. However by using this command var upperCase = require('upper-case') i can use upper-case module. So, what's the difference between module and package in this context. Are…
Haider Yaqoob
  • 1,725
  • 2
  • 11
  • 17
2
votes
1 answer

Node.js code works locally but does not work on AWS Lambda

I have a node.js function for AWS Lambda. It reads a JSON file from an S3 bucket as a stream, parses it and prints the parsed objects to the console. I am using stream-json module for parsing. It works on my local environment and prints the objects…
Vinay Dhavala
  • 31
  • 5
2
votes
2 answers

Typescript on node.js stream - incorrectly extends base class 'Transform'

import { Transform } from "stream"; export class TestStream extends Transform { constructor(options) { super(options); } write(data: any, enc: string, cb: Function) { return super.write(data,…
MD. Jahidul Islam
  • 635
  • 5
  • 20
2
votes
1 answer

process.stdout.write / process.stderr.write monkey-patch works in child process but not parent

So I have this simple monkey-patch on process.stdout.write / process.stderr.write const strm = fs.createWriteStream(logfile); const stdoutWrite = process.stdout.write; process.stdout.write = function () { …
Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
2
votes
1 answer

Node.js child.stdin.write doesn't work

when I try to run child process and put to it stdin some text it throws error. here is code of child process: import java.io.Console; public class HelloWorld { public static void main(String[] args) { System.out.println("started"); …
Dmytro Nalyvaiko
  • 1,664
  • 4
  • 16
  • 27
2
votes
1 answer

How to set a variable to the result of morgan (not to log it on the console) in Node.js

I do not understand the stream in node well. I try to save my app's request and response information to the mongodb. Firstly, I want to set a variable to the result of morgan: var apiInfo = morgan('dev') …
leon
  • 49
  • 6
2
votes
2 answers

calling an Rscript from node.js

I have been trying to execute an Rscript from my node.js server. tried to follow an example online, but i keep getting a null returned object or sometimes the process keeps running forever. I have mentioned the code snippet below. Thank…
apsara dhanasekar
  • 49
  • 2
  • 7
2
votes
1 answer

How can I read a stream of JSON objects into browser

Given a node.js style object stream as the example below shows, how can this be read by an http request on a webpage and processed: {"id":"one", "value":"the first object"} {"id":"two", "value":"the second object"} {"id":"three", "value":"the…
Fergie
  • 5,933
  • 7
  • 38
  • 42
1 2 3
…
13 14