0

i have a problem with high cpu usage when streaming file from Mongodb Gridfs to file system. I'm currently using gridfs-stream node.js module and the code is as follows:

//Main server file
app.get('/download/:name', crud.download)

//crud.js file
exports.download = function(req, res){
    var file = req.params.name
    var readstream = gfs.createReadStream(file);
    readstream.pipe(res);

}

On smaller files (~100mb and more) cpu usage is pretty low while on bigger files (~1GB and more) cpu goes to 90-100% and stays like that, slowing download speed significantly, and sometimes even blocking whole download. Also, is there a better way for handling downloads to file system from Gridfs? Would like to hear some opinions. Thanks.

Case09
  • 313
  • 6
  • 18
  • What exactly is taking up the CPU? Node, MongoDB? – robertklep Dec 03 '13 at 21:09
  • Thanks for the reply, node.js is taking about 65% of cpu, while mongo is using about 30% while download is in progress. Again this is only happening with larger files. I've tried on multiple machines and it is the same on every machine i've tried. – Case09 Dec 03 '13 at 21:21
  • I wonder about the efficiency of GridFS for such large files. Can't really find a lot about it sadly (most examples/benchmarks deal with small to tiny files). Are you running in a VPS? – robertklep Dec 03 '13 at 21:29
  • Nope, not running in a VPS, also i should mention that when uploading the same file i want to download, upload process uses a tiny amount of cpu on node.js side (~18%) and upload completes successfully. So the problem has something to do with the way gridfs-stream is utilizing file streaming from GridFs to file system, or at least that's what i think it is. – Case09 Dec 03 '13 at 21:41
  • I only have experience with small files and GridFS streaming, which always worked great. Not with larger files, so I won't be able to help much :( – robertklep Dec 03 '13 at 21:42

0 Answers0