0

I am streaming an xml file in, using fs.createWriteStream. For whatever reason, if the chunk coming in happens to break up xml closing tag, I am seeing a bunch of null characters being added to the data at the end of the chunk and it breaks the xml.

This may be caused since we have to replace all the xml reserved values(&lt,&gt, &#xD, etc) with the proper characters(<,>, etc.) after we stream the data file in. Any ideas on why this occurs and how to properly handle it? Here's the basic stream code:

    var writableStream = fs.createWriteStream(file_path + send_id + '.xml', { highWaterMark: 128 * 1024 });

  var options = {
    host: domain_name,
    path: f_path,
    method: 'POST',
    'Content-Type': 'text/html',
    'Content-Length':Buffer.byteLength(send_id)
    };   

   var req = http.request(options, (res) =>{
            res.setEncoding('utf8');
            res.on('data', (chunk) => {

                    writableStream.write(chunk);

            });
            res.on('end', () => {
                    console.log('No more data in response');
                     });
    });


    req.write(send_id);
    req.end();
Brian03
  • 23
  • 5
  • Not an answer, but have you considered using the pipe() function instead? Might make this a lot easier. – Evert Nov 13 '19 at 17:34
  • @Evert thanks for the suggestion, I'll definitely look into that. This is all pretty new to me so I wouldn't be surprised if there is a simpler way to do this. – Brian03 Nov 13 '19 at 17:37
  • Yeah if it were me i'd just pipe the response to the writable stream, you shouldn't need to deal with the individual chunks here I don't think unless i'm misinterpreting the use case. You probably also want to add an .on('error') handler as well to ensure you're handling any potential errors. – Sebastian Bou-Samra Nov 14 '19 at 12:25

0 Answers0