A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.
Questions tagged [nodejs-stream]
306 questions
0
votes
0 answers
Append to markdown file in S3
I'm trying to append a block of text to a markdown file stored in S3.
I know that S3 doesn't allow you to "append". I need to read the file's contents, append to them, and re-upload a file with the same name to replace it.
I'm not familiar with…
James
- 411
- 4
- 20
0
votes
1 answer
How to render my ejs view with response image from node js
I am using gridfs and mongoDB to store images in chunks. And whenever a user requests my server just sends a streamlined image file in response by piping.
Currently my code looks like this :
const download = async (req, res)=>{
try{
const fileName =…
Ketan Rathod
- 53
- 6
0
votes
0 answers
Where is memory leak in my JavaScript (NodeJS) code?
I have simple script to handle CSV file with size 10GB. The idea is pretty simple.
Open file as stream.
Parse CSV objects from it.
Modify objects.
Make output stream to new file.
I made following code, but it cause memory leak. I have tried a lot…
Jakeroid
- 205
- 2
- 11
0
votes
2 answers
JS - How to check if a string contains an exact phrase value in array
I am working on a script using Twitter's API and I am trying to find matches to exact phrases.
The API however doesn't allow queries for exact phrases so I've been trying to find a workaround however I am getting results that contain words from the…
730wavy
- 944
- 1
- 19
- 57
0
votes
1 answer
NodeJs writeStream storage in memory
I need to compress some files with archiver and then send them by email.
In archiver examples shows how to storage the created file to the fs but i need to temporarily storage it in memory to could send it.
Is there any way to do that?
sendEmail()…
Javier salvañá
- 191
- 2
- 16
0
votes
1 answer
hashing a streaming file before uploading to S3
I am trying to stream a file to S3 without storing the file to disk/ssd. I would like to have part of the hash of the file as a part of the filename when uploading to S3.
EDIT_v1:
Been trying to follow this post using busboy as the parser: Calculate…
M.Holmes
- 403
- 1
- 7
- 22
0
votes
1 answer
Only reading additional content in a file
I'm working on a project which requires reading a log file, The log file can get massive, so I only want to read additional data added to the file and not the old data to avoid potential performance issues.
I tried fs.createReadStream but it didn't…
0
votes
1 answer
How to compress Folder in nodeJS Mac without .DS_STORE
Using the folder-zip-sync npm library and other zipping libraries, The .zip file gets saved with an extra .DS_STORE file. How to zip without this file? Is there a setting I can turn off? How to go about this?
var zipFolder =…
0
votes
1 answer
NodeJS net.socket Pipe by condition/filter
I'm new to NodeJS. The existing net.socket pipe need to filter by condition to not to connect to "con2", my existing code as follow.
I found Transform and PipeLine methods and, so far I tried, the sample code not working for my scenario yet.
The…
Thet Thet
- 13
- 5
0
votes
0 answers
Download functionality using streams for large files in NodeJS
I am trying to implement a download functionality using streams in NodeJS.
In the code I am trying to simulate an endpoint that sends data in chunks, something similar to paginated data, for example chunks in size of 5000. Or to make it further…
0
votes
1 answer
Node.js pipe decipher file Error: Unsupported state or unable to authenticate data
I'm trying to encrypt/decrypt a file in stream with pipes. Encryption works, however when decrypting I get the error:
Error: Unsupported state or unable to authenticate data
at Decipheriv._flush (node:internal/crypto/cipher:160:29)
at…
GeoCap
- 505
- 5
- 15
0
votes
0 answers
Clone a read stream 3 times and wait for each stream to finish job before reading the next clone
I have create 3 read stream, that reads data form 3 tables in a database in JSON format. Now I need to complete three things with this stream and that too in sequence.
Create md5 cheksum of the stream.
save the stream in the filesystem in a JSON…
user5740953
- 169
- 2
- 16
0
votes
1 answer
How can I send an "end" event to Nodejs stream
Im' working with a pelias project and using package wof-admin-lookup to handle data that is read from a file.
There is a case, there is no valid data for being pushed to stream. The wof-admin-lookup will never end.
Here is my code:
const stream =…
Nguyen Hoang Vu
- 751
- 2
- 14
- 35
0
votes
1 answer
NodeJs AES File Decryption to a Stream
I am encrypting a file in C# and decrypting it in Node.js using AES.
The following example works fine but it writes the decrypted file to `output_dec.xml', but I want to write the decrypted output to a variable or stream.
const crypto =…
aadi1295
- 982
- 3
- 19
- 47
0
votes
1 answer
Extract matching row by comparing two CSV file in NodeJs
The scenario is I have two large CSV files csv1.csv and csv2.csv. In both the files, there is an email column and I have to read csv1.csv row by row and check if the email exists in csv2.csv and if matches write the row of csv2.csv in csv3.csv. I…
Himadri Ganguly
- 715
- 4
- 11
- 31