http.get request returns csv.gz file. After unzipping I would like to convert it to JSON and upload to mongoDB How to implement convertation function on stream?
My code is following:
const download = (url, dest) => {
const request = https.get(url, function(response) {
const decompress = zlib.createGunzip();
const file = fs.createWriteStream('./downloads/'+ dest);
response.pipe(decompress).on('data', (chunk) => {
console.log(chunk)
})
.pipe(file);
});
}
CSV to JSON will convert using that:
const csvToJson = (csv) => {
const [firstLine, ...lines] = csv.split('\n');
return lines.map((line) => firstLine.split(',').reduce(
(curr, next, index) => ({
...curr,
[next]: line.split(',')[index],
}),
{},
));
};
Calling code as follows:
linksToDownload.map((item) => {
if (item.value !== undefined) {
const URL = item.value;
const DEST = item.name + ".json"
download(URL, DEST);
}
})
Also, why nodemon restarts several times? printscreen of console
Thanks in advance for your help