1

I am currently trying to stream a very large CSV file (130k rows) to be converted to a JSON file. Which I can then iterate over later on.

The problem is csvtojson is adding a comma to the very last entry, making it invalid JSON to deal with.

var Converter = require("csvtojson").Converter;

var csvConverter = new Converter({
    constructResult: false,
    downstreamFormat: "array"
});

var readStream = require("fs").createReadStream(req.file.path);

var writeStream = require("fs").createWriteStream("csvData.json");

readStream
    .pipe(csvConverter)
    .subscribe((jsonObj, index) => {
           jsonObj.myNewKey = "some value";
    })
    .pipe(writeStream);

But the last JSON added to the array adds an extra comma at the end creating an invalid array/json.

Result is such:

[
  {Name: Bob},
  {Name: Sarah},
  {Name: James},
]

Expected result such:

[
  {Name: Bob},
  {Name: Sarah},
  {Name: James}
]

How could I make sure it doesn't add the comma on the last entry? Cheers

CyrisXD
  • 147
  • 9

0 Answers0