1

http.get request returns csv.gz file. After unzipping I would like to convert it to JSON and upload to mongoDB How to implement convertation function on stream?

My code is following:

  const download = (url, dest) => {
  const request = https.get(url, function(response) {
    const decompress = zlib.createGunzip();
    const file = fs.createWriteStream('./downloads/'+ dest);
    response.pipe(decompress).on('data', (chunk) => {
      console.log(chunk)
    })
    .pipe(file);
  });
  }

CSV to JSON will convert using that:

  const csvToJson = (csv) => {
    const [firstLine, ...lines] = csv.split('\n');
    return lines.map((line) => firstLine.split(',').reduce(
      (curr, next, index) => ({
        ...curr,
        [next]: line.split(',')[index],
      }),
      {},
    ));
  };

Calling code as follows:

  linksToDownload.map((item) => {
    if (item.value !== undefined) {
    const URL = item.value;
    const DEST = item.name + ".json"
    download(URL, DEST);
    }
  })

Also, why nodemon restarts several times? printscreen of console

Thanks in advance for your help

Igor
  • 21
  • 4

0 Answers0