A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.
Questions tagged [nodejs-stream]
306 questions
0
votes
1 answer
Trigger HTTP fileupload by reading a file from another multi part HTTP request
An upload end-point receives data from a client in multipart file. This file is again sent as a parameter to new HTTP call made from the same service.
//Client uploading a file code
form(enctype="multipart/form-data", action="/data/upload",…
AnandShiva
- 906
- 11
- 22
0
votes
1 answer
How to pipe data modified from a gunzip stream into a gzip stream?
I need to trigger, through an http request, a process where I download some data from S3, gunzip it, modify the stream, gzip it and send to another bucket in S3.
So far I was able to either:
Download
Gunzip
Modify (filter) the data
return the…
OJEP
- 26
- 5
0
votes
1 answer
Order of data piped/pumped through nodejs streams
I'm trying to write a nodejs code that read (audio) files and stream the content to a remote service (dialogflow). I'm having trouble ensuring the order of the chunks sent to the stream.
Most of the time everything seems to be in the right order,…
thammaknot
- 63
- 7
0
votes
0 answers
Copy data from CSV to postgres ignoring unique key violation
I'm trying to bulk insert data from CSV into the PostgreSQL database using the COPY SQL command. Usually the CSV file contains minimum of 10 thousand rows to maximum of one million rows. This CSV file contains queryId as one of the columns and this…
Akhil Vadrevu
- 57
- 8
0
votes
0 answers
Multiple socket connection for TCP server, How to scale using worker thread in Nodejs
I have following code, As there are N number of socket sending sequential stream.
how to scale this using worker threads in nodejs.
const Net = require('net');
const port = 8080;
const server = new Net.Server();
const processor =…
NRaj
- 121
- 8
0
votes
1 answer
Sharp.js not piping to stream.Writable?
Found on "sharp": "^0.26.0"
For some reason this does not work:
const readableStream = new stream.Readable({
objectMode: true,
read() {}
})
class FileSwitch extends stream.Writable {
constructor(options, folder) {
…
Bdyce
- 332
- 2
- 11
0
votes
1 answer
How do you catch stream transform errors?
This is the readable stream native definition
// This is the part where you do stuff!
// override this function in implementation classes.
// 'chunk' is an input chunk.
//
// Call `push(newChunk)` to pass along transformed output
// to the readable…
Alex
- 66,732
- 177
- 439
- 641
0
votes
1 answer
Where and How, under the hood in NodeJS, readable._read() is getting called?
I am so curious about the below program.This program push the letters 'a' through 'z', inclusive , into a readable stream only when the consumer requests. I understand from the docs that _read() implementation is called by internal Readable class…
user3656231
- 431
- 2
- 7
- 17
0
votes
0 answers
moving files from hard drive of a server to remote SFTP is an example of I/O operation?. If so , NodeJS can help here to speed up the process?
We have lots of pdf files (100K plus) in our server hard drive which we currently transfer to SFTP server using a java SFTP library. I have two questions here,
I believe moving files from hard drive of a server to remote SFTP is
an example of I/O…
user3656231
- 431
- 2
- 7
- 17
0
votes
1 answer
What is the best way to make a simple Video API in Node js?
Hey I've been on the search for quite a while now for a simple video API. I want to stream Videos on a simple page and implement them into my html. Any suggestions how I can make this work ?
AJ.beProgramming
- 87
- 4
0
votes
1 answer
Error When Sending PassThrough Stream in Response with NodeJS Express Server
I'm trying to pass a NodeJS Stream API PassThrough object as the response to a http request. I'm running an Express server and am doing something similar to the following:
const { PassThrough } = require('stream')
const createPassThrough = () => {
…
Colin
- 2,428
- 3
- 33
- 48
0
votes
1 answer
How to encrypt a stream in node js
How to encrypt and decrypt a stream in node js without saving the file locally or converting it into buffer.
if there is no way then pls provide memory efficient and less storage consuming way to encrypt and decrypt stream in node js so that i can…
Sherin Stephen
- 23
- 1
- 7
0
votes
1 answer
NodeJS Stream flushed during the Event Loop iteration
I'm trying to pipe one Stream Axios Response into multiple files. It's not working, and I can reproduce it with the simple code below:
Will work:
const { PassThrough } = require('stream')
const inputStream = new…
baruchiro
- 5,088
- 5
- 44
- 66
0
votes
4 answers
NodeJS: Using Pipe To Write A File From A Readable Stream Gives Heap Memory Error
I am trying to create 150 million lines of data and write the data into a csv file so that I can insert the data into different databases with little modification.
I am using a few functions to generate seemingly random data and pushing the data…
Will Park
- 55
- 2
- 7
0
votes
1 answer
Convert Webrtc track stream to URL (RTSP/UDP/RTP/Http) in Video tag
I am new in WebRTC and i have done client/server connection, from client i choose WebCam and post stream to server using Track and on Server side i am getting that track and assign track stream to video source. Everything till now fine but problem…
Zeeshan Younis
- 1
- 1
- 3