Questions tagged [node.js-stream]

A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.

There are many stream objects provided by Node.js. For instance, a request to an HTTP server and process.stdout are both stream instances.

Streams can be readable, writable, or both. All streams are instances of EventEmitter.

The stream module can be accessed using:

const stream = require('stream');

There are four fundamental stream types within Node.js:

  • Readable - streams from which data can be read (for example fs.createReadStream()).
  • Writable - streams to which data can be written (for example fs.createWriteStream()).
  • Duplex - streams that are both Readable and Writable (for example net.Socket).
  • Transform - Duplex streams that can modify or transform the data as it is written and read (for example zlib.createDeflate()).
205 questions
2
votes
1 answer

node.js, fetch a zip file and unzip it to ram

I need help, how to fetch a zip file and unzip it to memory, or maybe write it to harddisk. sample url var http = require("http"); var fs = require("fs"); var url = "http://www.caltrain.com/Assets/GTFS/caltrain/Caltrain-GTFS.zip" var file =…
vdj4y
  • 2,649
  • 2
  • 21
  • 31
2
votes
0 answers

Mocha tests for fs.readFile

My server code has a simple route in which I need to send a tiny thumbnail (jpeg) to the client for display. Here's a sample of the code: router.get('/thumbnail/:title/',function(req,resp){ let title = req.params.title; …
westandy
  • 1,360
  • 2
  • 16
  • 41
2
votes
1 answer

Convert PNG from GET request to a Readable Stream in Node.js

I'm trying to make a Facebook chat bot that can send cat pictures. I use a RESTful API to get the cat pictures. They are returned as raw png. The next and final step is to convert that image into a Readable Stream so the Facebook Chat API can send…
renxinhe
  • 41
  • 1
  • 4
2
votes
1 answer

How to write to js file at specific location on key value pairs?

I am writing to a js file using protractor as follows : index.js var outputFile = '../Actions/data_write.js'; var username = "someusername"; var password = "somepassword"; var fs = require('fs'); var text = "userCredentials : {username :…
2
votes
1 answer

Write to stream just before 'end' event is fired

I am using several child_process's with a Node.js parent process and I am piping all the stderr from the child processes to a file. Like so: const strmPath = path.resolve(projRoot + '/suman/stdio-logs/runner-stderr.log'); const strm =…
Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
2
votes
2 answers

Node.js How to use stdin stream to read user input after reading from shell pipe

After I receive data from shell pipe I need to ask user via prompt. But the application closes immediately after data is read from pipe. Hot to make it wait for user input? var readline = require('readline'); var data =…
storor
  • 21
  • 1
  • 2
2
votes
2 answers

Why isn't the 'finish' event firing in this Node.js code?

Newbie to Node here. I'm trying to download a .tar.xz file and extract it to a directory via the code shown below: var request = require('request'); var tar = require('tar'); var xz = require('xz'); function downloadRaw(url, callback) { return…
James Ko
  • 32,215
  • 30
  • 128
  • 239
2
votes
0 answers

How do I reset stdin after I have piped it to stdout?

Hi there first SO question! Anyway, I'm writing a little experiment to help get a better grasp of streams and more complex compositions combining streams & promises. However, I've run into an issue I'm currently unable to solve. If you run my…
2
votes
2 answers

Sending stream of data in response - node.js - Azure File service using node.js

I am using node.js to create file service for Azure File storage. I am using azure-storage-node (http://azure.github.io/azure-storage-node/) for this. I am trying to download a file from Azure file storage. Below is my code snippet // Download a…
Sniper
  • 2,412
  • 11
  • 44
  • 49
2
votes
1 answer

Remove headers from a piped HTTP stream

To give a short example of what I want to achieve, imagine we have an HTTP server already serving a given request: require('http').createServer(function(req, res) { var payload = new Buffer('Hello World\n', 'utf8'); res.writeHead(200, { …
Edwin Dalorzo
  • 76,803
  • 25
  • 144
  • 205
2
votes
2 answers

untarring files to S3 fails, not sure why

(new information below) I am trying to set up a lambda function that reacts to uploaded tgz files by uncompressing them and writing the results back to S3. The unzip and untar work fine, but uploading to S3…
russell
  • 660
  • 1
  • 10
  • 18
2
votes
1 answer

Append text to a node.js stream on the go

I am using fs.createReadStream() to read files, and then pipe them to the response. I want to add a small Javascript function when I'm serving HTML files. The only way I can think of is to read the file into a string, append the text to the string,…
Omar A
  • 435
  • 7
  • 14
2
votes
1 answer

JSON streaming with Oboe.js, MongoDB and Express.js

I'm experimenting with JSON streaming through HTTP with Oboe.js, MongoDB and Express.js. The point is to do a query in MongoDB (Node.js's mongodb native drive), pipe it (a JavaScript array) to Express.js and parse it in the browser with Oboe.js. The…
m_vdbeek
  • 3,704
  • 7
  • 46
  • 77
2
votes
1 answer

Calculating data bytes in node js

I have a vhd (disk) of size 500 MB, of which only 10 MB of data is written, followed by empty chunks and finally one more block of 10 MB towards the end. So , the total data present is just 20 MB out of 500 MB. I am trying to find a utility in…
user2630764
  • 624
  • 1
  • 8
  • 19
2
votes
3 answers

Why does the 'finish' event fire on a Node.js transform stream before _flush is finished?

I expect the 'finish' the callback to be triggered only once the callback passed to _flush has completed, but it's being triggered before _flush completes. Why? var stream = require('stream').Transform(); // Nothing interesting…
Mark Stosberg
  • 12,961
  • 6
  • 44
  • 49