How to avoid for await...of with csv-parse

Question

I am looking to be educated on this issue as I have spent a few days trying to resolve it on my own, to no avail.

I am using csv-parse to parse a CSV file.
I am using ESLint as my Linter
I am using the Airbnb JavaScript Style Guide plugin for ESLint
I am running this on the backend using NodeJS

My function is:

const { parse } = require('csv-parse');
const fs = require('fs');
const csvFile = 'myCsvFile.csv';

async function parseCsv(csvFile) {
  const records = [];
  const parser = fs.createReadStream(csvFile).pipe(parse({ delimiter: ',', columns: true }));
  for await (const record of parser) {
    records.push(record);
  }
  return records;

The function works well, however I am trying to abide by Airbnb's Style Guide, which does not like the for await...of loop as it is hitting me with the no-restricted-syntax violation.

I am curious on if there is a better way to write this to fall in line with Airbnb's Style Guide or, if this is one of those situations where it's OK to ignore the violation?

Well, does any of the airbnb reasoning that you linked resonate with you? — Bergi, Dec 19 '21 at 21:44
*Why? This enforces our immutable rule. Dealing with pure functions that return values is easier to reason about than side effects.* This doesn't seem to apply to your situation. — Emiel Zuurbier, Dec 19 '21 at 22:06
@Bergi, great question. No, it did not resonate with me, which is why I am asking. I’m just a few years into this journey and sometimes I need help seeing the forest through the trees. What one may see as easy, I still miss. — Jeremy M, Dec 19 '21 at 22:19
@JeremyM Then I'd recommend not to use that style guide (or at least not that rule). Imo, airbnb is a bit misguided and often prevents you from using modern syntax. — Bergi, Dec 19 '21 at 22:20
@EmielZuurbier In a way, it actually does, `records.push(record)` is a side effect. The problem is just that currently there are no helper methods on node streams (or async iterators) that would alleviate this - however with the [*iterator helpers* proposal](https://github.com/tc39/proposal-iterator-helpers/) you could simplify all that to just `const records = await parser.toArray()` — Bergi, Dec 19 '21 at 22:24

score 3 · Answer 1 · answered Dec 19 '21 at 22:08

The styleguide says:

11.2 Don’t use generators for now. Why? They don’t transpile well to ES5.

Fortunately if you're using a recent NodeJS version, you don't need to transpile down, and can use the engine's native support. For browsers this advice is also outdated soon.

num8er · Answer 2 · 2021-12-20T13:32:51.170

2

How about using events end returning promise?

const { parse } = require('csv-parse');
const fs = require('fs');
const csvFile = 'myCsvFile.csv';

async function parseCsv(csvFile) {
  return new Promise((resolve) => {
    const records = [];
    const stream = fs.createReadStream(csvFile);
    const parser = stream.pipe(parse({ delimiter: ',', columns: true }));
    
    parser.on('readable', () => {
      while (record = parser.read()) {
        records.push(record);
      }
    });

    let ended = false;
    const end = (error) => {
      if (error) {
        console.error(error.message);
      }

      if (!ended) {
        ended = true;
        resolve(records);
      }
    };

    parser.on('error', end);
    parser.on('end', end);
  });
}

also if You have node 15+ then try stream/promises example:

const { parse } = require('csv-parse');
const fs = require('fs');
const { finished } = require('stream/promises');
const csvFile = 'myCsvFile.csv';

async function parseCsv(csvFile) {
    const records = [];
    const stream = fs.createReadStream(csvFile);
    const parser = stream.pipe(parse({ delimiter: ',', columns: true }));

    parser.on('readable', () => {
      let record;
      while ((record = parser.read()) !== null) {
        records.push(record);
      }
    });

    await finished(parser);

    return records;
}

edited Dec 20 '21 at 13:32

answered Dec 19 '21 at 22:19

num8er

18,604
3
43
57

I'd really question the styleguide if replacing a 3 lines for loop with 20 lines of callbacks and manual promise resolution makes the code compliant ... – Jonas Wilms Dec 19 '21 at 22:24
@JonasWilms I tried to make code not to have async/awaits, I think that's reason to make it more compatible to old lts stuff :) – num8er Dec 19 '21 at 22:26
I was able to make it work as a Promise but I agree with @Jonas Wilms, it took a lot more code to make it work. I’m starting to question the Style Guide. This will shake my world though . I started this journey 3 years ago forcing myself to learn following the Airbnb style guide. It’s almost a religion for me. – Jeremy M Dec 19 '21 at 22:30
@JeremyM You can use `on_record` and simplify my code, https://csv.js.org/parse/options/on_record/ – num8er Dec 19 '21 at 22:38
1

@JeremyM if You're using node 15+, then please check example with stream/promises, I've updated my answer – num8er Dec 19 '21 at 22:58
@num8er It works well, except `while (record = parser.read())` triggers [no-cond-assign](https://eslint.org/docs/rules/no-cond-assign). Can't win with ESLint. – Jeremy M Dec 20 '21 at 13:21
@JeremyM updated my answer, please check – num8er Dec 20 '21 at 13:26

score 1 · Accepted Answer · answered Dec 20 '21 at 15:15

Based on advice given in the answers, I am going to ignore the Airbnb Style Guide and use the Async iterator method.

Final code:

const { parse } = require('csv-parse');
const fs = require('fs');
const path = require('path');
const debug = require('debug')('app:csv:service');
const chalk = require('chalk');

async function parseCsv(csvFile) {
  try {
    const records = [];
    const stream = fs.createReadStream(csvFile);
    const parser = stream.pipe(parse({ delimiter: ',', columns: true }));
    // eslint-disable-next-line no-restricted-syntax
    for await (const record of parser) {
      records.push(record);
    }
    return records;
  } catch (error) {
    debug(`${chalk.red('Failed')} to parse CSV`);
    debug(`${chalk.red('ERROR:')} ${error}`);
    throw error;
  }
}

It may be time to find a new Style Guide to follow. Thank you to num8er for the code advice (I took one of your ideas to make my code a little more readable).

How to avoid for await...of with csv-parse

3 Answers3