0

I got library xml-stream which reads xml from stream and emits 'endElement' event on every interesting tag, under the hood it uses node-expat which is also written event way

On every tag I want to save it to database with bookshelf/knex ORM which use promises

The problem I've met is finding out when to end the process; lib emits event 'end' when it finish reading from stream but writing to db is not yet finished so process can't exit

XML is huge so I can't wait to read every tag and then resolve write promisses

does anybody know a better way to do something like that other way than with timeout:

parseSaveCallback => {
  var stream = fs.createReadStream(__dirname + '/xml/' + xmlFile);
  var xmlStream = new XmlStream(stream);

  var streamEndDeferred = Promise.defer()
  xmlStream.on('end', function(){
    streamEndDeferred.resolve()
  })

  var writeTimeout      
  var writeTimeoutDeferred = Promise.defer()
  var refreshTimeout = function() {
    clearTimeout(writeTimeout)

    writeTimeout = setTimeout(function(){
        writeTimeoutDeferred.resolve()
    }, 5000)
  }

  xmlStream.preserve(xmlElement)
  xmlStream.on('endElement: ' + xmlElement, function (item) {
    parseSaveCallback(item)
    .then(function(){
      refreshTimeout()
    })
  })

  return Promise.join(
    streamEndDeferred.promise,
    writeTimeoutDeferred.promise
  )
}
//.then(process.exit) ???

or is there any library for reading and saving XML from stream compatibile with promises? further XML will be read from HTTP and will contain GB's of data

madeuz
  • 201
  • 2
  • 3
  • For saving an XML stream, you will need a stream writer libary. You can surely use it with promises if you want, just as you have done with that `streamEndDeferred`. – Bergi Feb 13 '15 at 10:28
  • but I want to write that XML stream to MySQL, not another XML – madeuz Feb 13 '15 at 13:40

0 Answers0