0

I am trying to upload a large csv file of 10mb using csv parser and form data in node js while sending a request to an api with axios. My code is the following:

//insert new data
    else if (`${flags.newdata}` !== "undefined" && `${flags.source}` !== "undefined" ){
        var csv=require('csv-parser');
        var FormData=require('form-data');
        const form_data = new FormData();
        var data = [];
        form_data.append("file", fs.createReadStream(`${flags.source}`)).pipe(csv()).on('data',function(row){ data.push(row) }).on('end',function(){console.log('Data Loaded')});

        /*const request_config = {
            headers: {
                "Authorization": token,
                "Content-Type": "multipart/form-data"
        },
        data: form_data
        };*/

        return axios.post('https://localhost:8765/energy/api/Admin/' + `${flags.newdata}`, form_data);
    }

I get the following error:

TypeError: Cannot read property 'pipe' of undefined

I have read similar questions but none helped. How can I solve this?

  • Have you checked what the append method returns? In case you didn't it does not return anything. So chaining won't work here. Try `form_data.append('file', ...); form_data.pipe(...)` Unfortunately i don't think it will work this way either. The pipe method is asynchronous and you are calling `axios.post` synchronously before csv is finnished. You might need to put the `axios.post` call to the `on('end', ...` event handler. – Molda Feb 26 '20 at 16:28
  • @Molda it did work but I get the error: request body larger than bodylength limit –  Feb 26 '20 at 16:30
  • 1
    try to increase axios option maxContentLength or maxBodyLength. The default limit should be 10MB(not sure) so if your data exceeds that try `maxContentLength or maxBodyLength: 20000000` that's 20MB in bytes. I'm not sure which is for what so just try both. https://stackoverflow.com/a/58656297/3284355 – Molda Feb 27 '20 at 08:13

1 Answers1

0

Looks like you want to pipe not a stream but result of append() but append return undefined. Please use pipe like this:

 var readStream = fs.createReadStream(`${flags.source}`);

 readStream.on('open', function () {

   readStream.pipe(csv());

 });
 readStream
   .on('data',function(row){ data.push(row) })
   .on('end',function(){console.log('Data Loaded')});
Faust Life
  • 61
  • 5