parsing writable stream data to JSON?

Question

I am writing received data (CSV) to a local txt file and want to parse it as JSON before it writes into my local file, any suggestions?

 .then(async data => {
    const filteredFile = data.filter(file => {
      let currentDate = moment();
      let CurrentTimeMinusFive = moment().subtract(5, "minutes");

      let allAccessTimes = file.accessTime;
      let parsedAccessTimes = moment(allAccessTimes);
      let filteredTime = moment(parsedAccessTimes).isBetween(
        CurrentTimeMinusFive,
        currentDate
      );
      // console.log(filteredTime);
      return filteredTime;
    });
    console.log(filteredFile);

    let localPath = "/Users/ME/Desktop/DOWNLOADEDSFTP/data.json";

    for (const file of filteredFile) {
      let name = file.name;
      let doc = await sftp.get(
        `FILE/${name}`,
        localPath // parse before writing here
      );
    }
  })
  .then(() => {
    sftp.end();
  })

i tried using the buffer to get the data but i just get back the local file path?

let bufferOne = Buffer.from(doc);
  let json = JSON.stringify(bufferOne);
  let bufferOriginal = Buffer.from(JSON.parse(json).data);
  console.log(bufferOriginal.toString("utf8"));

i've looked into csvtoJSON packages but they dont seem to be able to solve this? hope im wrong, thanks in advance to the best dev community!

CSV data is by definition, not JSON. So, it has to be converted to JSON either manually or with a library that knows how to do that. You can't call `JSON.parse()` or `JSON.stringify()` on CSV data directly. You have to read the column names, save those and then turn each row into an object with the appropriate property names (from the column titles) and the values from each CSV line so you end up with an array of Javascript objects which you could then call `JSON.stringify()` on. JSON does not naturally lend itself to streaming, though some packages have been written to do it. — jfriend00, Feb 22 '20 at 07:17

parsing writable stream data to JSON?

0 Answers0