I have a function where I'm trying to take an existing txt file that is delimited by tabs and transform it into a CSV. I'm using the "csv-parser" library as a transformer. Using this method I have a file that's a little over 6 MB in size, with about 1200 lines to start.
function convertToCSV(arr) {
const array = [Object.keys(arr[0])].concat(arr);
return array
.map((val) => {
return Object.values(val).toString();
})
.join("\n");
}
fs.createReadStream(filenameTXT, {encoding:'utf-8'})
.pipe(csv({ separator: "\t" }))
.on("data", (data) => results.push(data))
.on("end", () => {
console.log(results.length);
fs.writeFileSync("new.csv", convertToCSV(results));
});
For some reason this function only receives about half of the lines I'm trying to transform (console.log shows 590), leaving the rest untransformed which means these lines get missed when I upload them using another API that expects a CSV format. My goal is to essentially automate what Excel or Google sheets does when it comes to transforming a txt to a csv. Any help or a better method of doing this would be much appreciated.