Stack Overflow
Stack Overflow

Questions tagged [node.js-stream]

A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.

There are many stream objects provided by Node.js. For instance, a request to an HTTP server and process.stdout are both stream instances.

Streams can be readable, writable, or both. All streams are instances of EventEmitter.

The stream module can be accessed using:

const stream = require('stream');

There are four fundamental stream types within Node.js:

  • Readable - streams from which data can be read (for example fs.createReadStream()).
  • Writable - streams to which data can be written (for example fs.createWriteStream()).
  • Duplex - streams that are both Readable and Writable (for example net.Socket).
  • Transform - Duplex streams that can modify or transform the data as it is written and read (for example zlib.createDeflate()).
205 questions
9
votes
2 answers

node.js stream array of json to response

I have a REST method that should return a JSON array with certain elements that are read from mongodb (using mongoose). It should be very simple (in the real case the find method has arguments, but that's no the problem):…
richardtz
  • 4,993
  • 2
  • 27
  • 38
9
votes
1 answer

Extra stdio streams for node.js process

The node.js API documents using an extra stdio (fd=4) when spawning a child process: // Open an extra fd=4, to interact with programs present a // startd-style interface. spawn('prg', [], { stdio: ['pipe', null, null, null, 'pipe'] }); That stdio…
Blake Regalia
  • 2,677
  • 2
  • 20
  • 29
9
votes
2 answers

Node.js - How to get stream into string

I have got stream and I need to get stream content into string. I stream from internet using http.get. I also write stream into file, but I don't want to write file and after that open the same file and read from it... So I need to convert stream…
user2316602
  • 622
  • 4
  • 9
  • 25
8
votes
1 answer

Node js file system: end event not called for readable stream

I'm trying to extract a .tar file(packed from a directory) and then check the names of the files in the extracted directory. I'm using tar-fs to extract the tar file and then use fs.createReadStream to manipulate the data. Here's what I've got so…
odieatla
  • 1,049
  • 3
  • 15
  • 35
8
votes
2 answers

"readable" event occurs twice

var fs = require('fs'); var file = fs.createReadStream('./zeros.txt'); var dataSize = 0; file.on('readable', function () { var data = file.read(10); console.log('readable size = ', data.length); console.log(data.toString()); }); Thie…
makoven
  • 157
  • 1
  • 5
8
votes
2 answers

Node.js net library: getting complete data from 'data' event

I've searched around, and either can't find the exact question I'm trying to answer, or I need someone to explain it to me like I'm 5. Basically, I have a Node.js script using the Net library. I'm connecting to multiple hosts, and sending commands,…
coding_hero
  • 1,759
  • 3
  • 19
  • 34
7
votes
1 answer

Node.js PassThrough stream not closing properly?

I'm curious about my PassThrough stream and why it isn't closing after a resource I pipe it to closes. I'm using it as a mediator, one resource needs a ReadableStream and I need to pass the user a WriteableStream to allow them to write the…
four43
  • 1,675
  • 2
  • 20
  • 33
7
votes
1 answer

Node fs.readstream() outputs instead of readable data

I'm reading a large XML file (~1.5gb) in Node Js. I'm trying to stream it and do something with chunks of data, but I'm finding it difficult to understand the documentation. My current simple code is: var fs = require('fs'); var stream =…
JVG
  • 20,198
  • 47
  • 132
  • 210
7
votes
2 answers

Troubleshooting Error: connect ECONNREFUSED in nodejs stream-adventure tutorial

I've been working through the learnyoujs and stream-adventure tutorials: https://github.com/substack/stream-adventure https://github.com/rvagg/learnyounode#learn-you-the-nodejs-for-much-win I've gotten all the way through the first set and most of…
John Reeve
  • 142
  • 1
  • 6
6
votes
1 answer

How to wait for all node streams to finish/end?

I have a child Node process that runs on a CRON once every 24 hours. When the process begins, it reads some queued up data and shoves that data down some transform stream. This stream then acts as an inverse multiplexer and splits the stream into…
puopg
  • 563
  • 7
  • 17
6
votes
3 answers

How do I test if a stream has ended?

I have a stream that may or may not have already ended. If it has ended, I don't want to sit there forever waiting for the end event. How do I check?
cbnz
  • 656
  • 1
  • 9
  • 19
6
votes
2 answers

Must I repeatedly call readable.read() within a readable event handler?

Suppose I have created a transform stream called Parser which can be written to like a normal stream but is read from as an object stream. I am using the readable event for the code that uses this transform stream: var parser = new…
Brad
  • 159,648
  • 54
  • 349
  • 530
5
votes
1 answer

Nodejs stream pausing (unpipe) and resume (pipe) intermediate pipe

I need to "pause" a readable stream for a certain number of seconds and resume it again. The readable stream is being piped to a transform stream, so I cannot use the regular pause and resume methods, I had to use unpipe and pipe. In the transform…
smac89
  • 39,374
  • 15
  • 132
  • 179
5
votes
3 answers

How to ensure asynchronous code is executed after a stream is finished processing?

I have a stream that I process by listening for the data,error, and end events, and I call a function to process each data event in the first stream. Naturally, the function processing the data calls other callbacks, making it asynchronous. So how…
modulitos
  • 14,737
  • 16
  • 67
  • 110
5
votes
2 answers

Node.js stream into zip archive and stream zip archive as response to client

Is it possible with Node.js streams to construct a zip archive and serve that zip archive to a client/user via a response to an HTTP GET request while it's being created? I'm searching for a solution that preferably avoids buffering the entire zip…
Casey Flynn
  • 13,654
  • 23
  • 103
  • 194
1
2
3
…
13 14