Why is the data not being written to the file?

Question

I am trying to create a file with dummy data. Since the file will be huge with 32^5 data points, I was using the write-stream. But I cannot see any data being written to the file. What could be the reason for this?

const faker = require('faker');
const fs = require('fs');
const os = require('os');

const MAX_DATA_POINTS = Math.pow(32, 5)

const stream = fs.createWriteStream('sample-data.csv', { flags : 'a' });

for(let i = 0; i < MAX_DATA_POINTS; i++) {
        console.log(i)
        stream.write(`${faker.name.findName()}, ${i} ${os.EOL}`);
}

console.log(`Written ${MAX_DATA_POINTS} .. `);

score 0 · Answer 1 · answered Apr 24 '19 at 10:29

The write method returns false if the stream wishes for the calling code to wait for the drain event to be emitted before continuing to write additional data and true otherwise. It is quite possible that the stream is not draining, and calls to write() are just buffering chunks and returning false.

You will need to wait till all the buffered chunks are drained (accepted for delivery by the operating system). When that happens, the drain event will be emitted.

Note: It is recommended that once write() returns false, no more chunks be written until the 'drain' event is emitted.

You could modify your dummy-csv file creator code in the way I have given.

const faker = require('faker');
const fs = require('fs');
const os = require('os');

const MAX_DATA_POINTS = Math.pow(32, 5);
let c = 0;

const stream = fs.createWriteStream('sample-data.csv', { flags : 'a' });

function write() {

    let ok = true;
    do {
        c++;
        ok = stream.write(`${faker.name.findName()}, ${c} ${os.EOL}`);
    } while (c < MAX_DATA_POINTS && ok);

    if(c < MAX_DATA_POINTS) {
        stream.once('drain', write)
    }

}


write()

Why is the data not being written to the file?

1 Answers1