What is the difference between the parsing and formatting methods in fast-csv for Node?

Question

My understanding is that they both create writeable streams, but I'm unsure of what the point of parsing the data is if you just go and format it afterwards like in the example below.

fs.createReadStream(path.resolve(__dirname, 'assets', 'snake_case_users.csv'))
    .pipe(csv.parse({ headers: true }))
    .pipe(csv.format({ headers: true }))
    .transform((row, next) => {
        User.findById(row.id, (err, user) => {
            if (err) {
                return next(err);
            }
            return next(null, {
                id: row.id,
                firstName: row.first_name,
                lastName: row.last_name,
                address: row.address,
                // properties from user
                isVerified: user.isVerified,
                hasLoggedIn: user.hasLoggedIn,
                age: user.age,
            });
        });
    })
    .pipe(process.stdout)
    .on('end', () => process.exit());

score 0 · Answer 1 · answered Jul 17 '20 at 22:21

I think the reason for the misunderstanding is that the example you've given is a bit wrong. The actual code should have the format stream piped after the transform.

The code should look a bit like this:

fs.createReadStream(path.resolve(__dirname, 'assets', 'snake_case_users.csv'))
    .pipe(csv.parse({ headers: true }))
    .pipe(csv.transform((row, next) => {
        User.findById(row.id, (err, user) => {
            if (err) {
                return next(err);
            }
            return next(null, {
                id: row.id,
                firstName: row.first_name,
                lastName: row.last_name,
                address: row.address,
                // properties from user
                isVerified: user.isVerified,
                hasLoggedIn: user.hasLoggedIn,
                age: user.age,
            });
        });
    }))
    .pipe(csv.format({ headers: true }))
    .pipe(process.stdout)
    .on('end', () => process.exit());

The idea of this is:

read file -> parse to objects -> transform object -> format to string -> print to stdout.

I also corrected the code so that is has the transform based on csv.transform.

See more here in their API docs.

What is the difference between the parsing and formatting methods in fast-csv for Node?

1 Answers1