A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.
Questions tagged [nodejs-stream]
306 questions
2
votes
2 answers
Firebase function Node.js transform stream
I'm creating a Firebase HTTP Function that makes a BigQuery query and returns a modified version of the query results. The query potentially returns millions of rows, so I cannot store the entire query result in memory before responding to the HTTP…
Robin
- 105
- 2
- 15
2
votes
1 answer
Consuming RabbitMQ with Node - limit parallel processing
I've got a RabbitMQ queue that might, at times, hold a considerable amount of data to process.
As far as I understand, using channel.consume will try to force the messages into the Node program, even if it's reaching its RAM limit (and, eventually,…
John Smith
- 3,863
- 3
- 24
- 42
2
votes
0 answers
How to get video chunks with Axios
So far I have the backend on nodejs which gives me the video I request by chunks.
I've been trying to get the video url and place it into a video tag using axios and responseType as blob, sending a range header.
This is the axios code:
await…
Chris Chong
- 187
- 1
- 11
2
votes
1 answer
How to make a function with events and pipe in nodejs
I'm writing a nodejs library in typescript, the main scope of this library is gonna be downloading stuff from a given url, what I'd like it to do is to be used like this
import library from 'library'
library('https://www.example.com')
…
Federico Morrone
- 83
- 8
2
votes
1 answer
How to add an instance of Vinyl to gulp stream?
I would like to add a new Vinyl file to the stream of a gulp task.
function task_add_vinyl_file(cb){
const file = new Vinyl({
cwd: '/',
base: '/test/',
path: '/test/file.js',
contents: Buffer.from('var x = 123')
…
chepe263
- 2,774
- 22
- 38
2
votes
1 answer
Node js Stream file without saving to memory
I am building an API that needs to accept file uploads. So a user can POST a file to an endpoint, the file will be sent to a virus scan, then if it's clean will be sent to storage (probably S3). So far I have achieved this with one issue: The files…
Barry Piccinni
- 1,685
- 12
- 23
2
votes
1 answer
Is it possible to use a Cloudflare Service Worker as a proxy web server?
this is just a beginner level question looking for some advice, which the following may misuse some key terms as well because of lacking knowledge, but hopefully can deliver the key message to all of you:
Background:
my company is hosting a website…
Donkie
- 51
- 2
- 4
2
votes
0 answers
Reading a ZIP archive from S3, and writing uncompressed version to new bucket
I have an app where user can upload a ZIP archive of resources. My app handles the upload and saves this to S3. At some point I want to run a transformation that will read this S3 bucket unzip it, and write it to a new S3 bucket. This is all…
Robert Lemiesz
- 1,026
- 2
- 17
- 29
2
votes
1 answer
8192 limited data size in NodeJS request
I'm trying to fetch json from an api but only half of the response is received. So how to get the full response?
var request = require('request');
var url_to_check = 'http://example.com/api/test';
request.get(url_to_check).on('data', function(data)…
Alice
- 55
- 3
2
votes
3 answers
Is there any way to save mulaw audio stream from twilio in a file
I am using Twilio voice stream feature and i don't want to use Twilio record functionality. When Twilio starts sending voice stream to my server i want to store it into disk as an audio file in realtime.
shashank
- 439
- 1
- 4
- 17
2
votes
0 answers
Handling deletion of a file while the same file is being modified or deleted in a WriteStream
When opening a file using fs.createWriteStream, how do you handle the case where the underlying file is deleted (eg by a 3rd party, such as logrotate)?
None of the normal events seem to fire in this case.
For example:
const stream =…
Alec
- 2,432
- 2
- 18
- 28
2
votes
0 answers
How to correctly pipeline Node.js duplex streams?
I have multiple duplex streams with asynchronous flush methods, but pipeline does wait for only one of them.
Working example code:
const { pipeline, Readable, PassThrough, Writable } = require('stream')
const { promisify } = require('util')
const…
user2219435
- 249
- 1
- 3
- 10
2
votes
0 answers
How to properly read multiple zlib streams from one base stream (Git Packfile) in NodeJS?
I'm creating a NodeJS module to interact with git repositories and I'm trying to read Packfiles coming from public git hostings like GitHub.
The problem is, Packfiles are made of many zlib streams but I can't figure out how to:
inflate (uncompress)…
Max1Truc
- 124
- 7
2
votes
1 answer
Implementing backpressure in a transform stream
I was reading these docs:
https://nodejs.org/ja/docs/guides/backpressuring-in-streams/
It says:
Readable.pipe(Transformable).pipe(Writable);
"Backpressure will be automatically applied, but note that both the
incoming and outgoing highWaterMark…
user11810894
2
votes
1 answer
How to stream range http requests from buffers in NodeJS
Testing http range requests in node, when we stream from
fs.createReadStream('index.html').pipe(res)
to a http res of a http server, the http client from node accepts it.
But, when I pipe a Stream of a Buffer, with:
const content =…
Paulo Coghi
- 13,724
- 14
- 68
- 90