Stack Overflow
Stack Overflow

Questions tagged [node.js-stream]

A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.

There are many stream objects provided by Node.js. For instance, a request to an HTTP server and process.stdout are both stream instances.

Streams can be readable, writable, or both. All streams are instances of EventEmitter.

The stream module can be accessed using:

const stream = require('stream');

There are four fundamental stream types within Node.js:

  • Readable - streams from which data can be read (for example fs.createReadStream()).
  • Writable - streams to which data can be written (for example fs.createWriteStream()).
  • Duplex - streams that are both Readable and Writable (for example net.Socket).
  • Transform - Duplex streams that can modify or transform the data as it is written and read (for example zlib.createDeflate()).
205 questions
5
votes
2 answers

NodeJS parseStream, defining a start and end point for a chunk

Confused by Node's filesystem parsing. Here's my code: var fs = require('fs'), xml2js = require('xml2js'); var parser = new xml2js.Parser(); var stream = fs.createReadStream('xml/bigXML.xml'); stream.setEncoding('utf8'); stream.on('data',…
JVG
  • 20,198
  • 47
  • 132
  • 210
5
votes
1 answer

efficiently serialise (and read) int array from nodejs

I'm considering building an application in nodejs which would need to stream large (>GB) files containing an array of integers. Crucially the array needs to be serialised optimally, so not ascii based, ideally using 8 bits for smaller integers…
Nat
  • 2,689
  • 2
  • 29
  • 35
4
votes
1 answer

how pipe() will performing read and write operations in node js?

var fs = require("fs"); var readStream= fs.createReadStream('input.txt'); var writeStream=fs.createWriteStream('output.txt'); readStream.pipe(writeStream); console.log("hello...world"); In the above program, readStream and writeStream…
Krishna Mohan
  • 159
  • 1
  • 11
4
votes
1 answer

node.js createWriteStream doesn't create new file on Heroku

I have following code that works fine on my localhost running node.js 0.12.0. The code creates a new file, and copy data from readable, but it doesn't create new file on Heroku. var output =…
ZoM
  • 506
  • 1
  • 6
  • 15
4
votes
1 answer

Node.js - Error: write EPIPE code: 'EPIPE

I keep getting EPIPE error, on stdin stream and I can't find a reason: This is my code: var checkFile = function(data, callback){ var child_process = spawn('ffprobe', ['-print_format', 'json', '-show_format', 'pipe:0']); var stdInError =…
MeV
  • 3,761
  • 11
  • 45
  • 78
4
votes
3 answers

How do I close a stream that has no more data to send in node.js?

I am using node.js and reading input from a serial port by opening a /dev/tty file, I send a command and read the result of the command and I want to close the stream once I've read and parsed all the data. I know that I'm done reading data by and…
ShaneH
  • 640
  • 8
  • 15
3
votes
1 answer

Fetching Large File For Processing Using Node.js

I have a Node.js application that needs to fetch this 6GB zip file from Census.gov and then process its content. However when fetching the file using Node.js https API, the downloading stops at different file size. Sometime it fails at 2GB or 1.8GB…
Sam
  • 57
  • 7
3
votes
1 answer

Encrypt a stream after a transform using openpgp

I'm using the below pipeline to stream data from Aurora, transform it to csv, and send it to S3. Readable knex stream: const getQueryStream = (organizationId) => db.select('*') .from('users') .where('organization_id', organizationId) …
navig8tr
  • 1,724
  • 8
  • 31
  • 69
3
votes
1 answer

reopen write stream after passing stream.end()

Is it possible reopen a closed stream after calling stream.end()? My requirement is to use streams to write in chunks and the whole process is called inside a scheduler, so once the stream is closed and the next time scheduler is called the said…
PrivateOmega
  • 2,509
  • 1
  • 17
  • 27
3
votes
0 answers

Recording MediaStream server-side (with video and audio) from WebRTC to a file

I am designing an application which aim is to stream video (with audio) from camera (mobile application) and save it on the disk (on the the server). Everything with the communication between client and server is ok. I am using WebRTC. One of the…
Pear Team
  • 31
  • 2
3
votes
1 answer

Writing and reading to a file using streams

The below code writes to a file whatever I type in the console. It also simultaneously reads from the same file and displays whatever that's in the file. Everything I type in the console is saved in the file. I manually went and checked it. But…
NewbieCoder
  • 105
  • 9
3
votes
0 answers

Pipe is not a function

I have a function that returns a stream and I want to pipe that value to another function then return it in an Express response like so: const { Duplex } = require('stream'); const request = require('request'); const example = async function…
loganhuskins
  • 1,399
  • 3
  • 15
  • 33
3
votes
2 answers

How to get Gulp4 to wait for file write tasks to compete?

I'm trying to write a workflow with Gulp 4 (see below for specific version info) that will watch a local folder for an .html file strip multiple tables out into individual .html files per table convert said tables into .csv for further…
Sn3aKyGuY
  • 33
  • 4
3
votes
2 answers

How to consume MTOM SOAP web service in node.js?

I need to download or process a file from a soap based web service in node.js. can someone suggest me on how to handle this in node.js I tried with 'node-soap' or 'soap' NPM module. it worked for normal soap web service. But, not for binary steam or…
Veer R
  • 129
  • 1
  • 9
3
votes
1 answer

How to pipe output of a node CLI program to shell?

I basically want to do something like this: $ my-node-cli | less Note that less is just an example. I need it to work with any other *nix command. More about the use case: I wrote a node CLI package that searches some online resource…
Marko Bonaci
  • 5,622
  • 2
  • 34
  • 55
1 2
3
…
13 14