A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.
Questions tagged [nodejs-stream]
306 questions
2
votes
1 answer
Data handler does not fire, even though connection is made
Seeing this strange problem where no data is being received by request to http server. I am just trying to stream some JSON line by line.
const http = require('http');
// server code here:
const s = http.createServer((req, res) => {
…
Alexander Mills
- 90,741
- 139
- 482
- 817
2
votes
1 answer
How streams API data flows in NodeJS?
I am still not clear on how the data flows between stream in pipe. Most examples are using file or built in generator as starting point, then finished in another file or output. But most of them didn't show anything about how to make our own…
Magician
- 1,944
- 6
- 24
- 38
2
votes
1 answer
Issue with loading chunks into the client
I'm trying to create a readableStream through which I would send chunks of data ( html in this case ). The problem is that even if I manage to get a chunk of data the res.write() method does nothing and just returns false if I log it.
I wonder why…
anathrax
- 91
- 1
- 1
- 11
2
votes
0 answers
sent file using axios using passthrough stream module in nodejs
Imports
const ffmpegPath = require('@ffmpeg-installer/ffmpeg').path;
const FfmpegCommand = require('fluent-ffmpeg');
const fs = require('fs');
const path = require('path');
const streamNode = require('stream');
const FormData =…
Harikrushna Patel
- 413
- 4
- 10
2
votes
1 answer
Writable Stream empties the file but do not write expected content in nodeJS
This is my code to get an array of records and write them as CSV string one by one in a file. I cannot get the writable stream to write in the file. It just erases whatever is in the file and does not write the new data in.
const writeStream =…
Chapo
- 2,563
- 3
- 30
- 60
2
votes
1 answer
filter bad json values from json in a memory efficient way
We have some bad json we're trying to parse. Unfortunately it isn't valid json as it returns unquoted NaN's in the payload.
We are switching from the long deprecated request library to axios. This seems to have doubled our memory usage of our…
xenoterracide
- 16,274
- 24
- 118
- 243
2
votes
1 answer
NodeJS Streams: Readable.pipe() doesn't send data immediately
From what I understood about streams, Stream.Readable.pipe() should pipe the data as soon as it receives it.
I am trying to implement my own streams but the output is not as expected.
const { Writable, Readable } = require("stream");
const writable…
Abhay
- 21
- 1
2
votes
1 answer
how to assemble a stream pipleine with node-formidable fileWriteStreamHandler?
I'm trying to upload a file to S3 using the node-formidable method fileWriteStreamHandler.Before the upload to S3 I want to create a hash of file. This means implementing a stream pipe that first pass the data through a hash then passes that data to…
M.Holmes
- 403
- 1
- 7
- 22
2
votes
0 answers
Convert pdf stream into buffer
I've a stream of pdf file which I need to convert into a buffer and I am tying to read the stream using stream.on but its not working and I keep getting error
UnhandledPromiseRejectionWarning: TypeError: stream.on is not a function
My stream…
Ehsan Nissar
- 643
- 2
- 13
- 35
2
votes
1 answer
Downloading large files using nodejs piped stream causes huge memory usage and OOM Error
I am using node js to download large files(300MB) from a server and pipe the response to a file write stream. As far as I understand pipes in nodejs, the data flow is managed by node and I don't have to consider draining and other events. The issue…
Dipanshu
- 39
- 5
2
votes
5 answers
Callback and event emitter functionality using NodeJS
Programme language is NodeJS
Steps ToDo:
1. The variable input has the input value. Extract n1 and n2 from the input.
2. Write a function to find the sum of all the multiples of n1 and n2, below and including 1000. This function should log the…
Coder-Neel
- 51
- 1
- 6
2
votes
0 answers
how to download partial content from remote video url in nodejs?
I have the following code to download a specific part (range) of a video using nodejs http.get(),
const fileUrl = 'https://www.example.com/path/to/video.mp4'
const fs = require('fs')
const http = require('https')
const fileName = 'video.mp4'
const…
ali hadi
- 21
- 1
2
votes
2 answers
Node.Js async iterator over stream pipeline
I have the following pipeline:
readFile > parseCSV > otherProcess
The readFile is the standard Node.Js createReadStream, while the parseCSV is a Node.js transform stream (module link).
I want to iterate through a csv file line by line and handle a…
Alessandro Staffolani
- 335
- 1
- 3
- 17
2
votes
1 answer
AWS: Can not download file from SSE-KMS encrypted bucket using stream
I have a bucket with enabled encryption configured:
encryption key type: AWS Key Management Service key (SSE-KMS)
using AWS managed key (aws/s3).
Bucket encryption config:
I have nodejs project using streams to upload/download files on S3…
Crowley
- 169
- 6
2
votes
0 answers
Request.pipe() JPEG streams
I'm working on a nodejs app that works as a middleware for ZoneMinder. In short, my goal is to mask all ZoneMinder api so that the client doesn't know if I'm using ZM or not.
Everything work fine, but there is one thing that bugs me. There is a…
ThienLD
- 741
- 4
- 14