0

I am storing a file in memory using multer, the idea is to read the buffer and check the contents of the file line by line, the moment a line does not pass validation I want to discard otherwise save to disk.

The file is a csv and I have tried createReadStream but it fails with buffer so I have used node stream.

The problems I am having given this implementation is that I can not return and stop anything from the Readable stream

Readable.from(buffer)
  .pipe(csv())
  .on('data', (data) => {
    if (data.value > 10) {
      return 'Stop the reading of the file I dont want it anymore'
    }
  })
Álvaro
  • 2,255
  • 1
  • 22
  • 48
  • I gather that the file upload has already been completed (by multer) before you apply the validation. Is there a motivation for using a stream to process the data when it’s already in memory? It appears that a simple toString().split() might just as well do the trick? – jorgenkg Jul 30 '21 at 04:06
  • Regarding the streams API: using the event driven approach “on(data)” does not throttle processing of the stream data. Instead, you should look into eg writing your own stream.Writable or piping the data to a async function using stream.pipeline. – jorgenkg Jul 30 '21 at 04:09

0 Answers0