I have a web-server running on an embedded linux running on an armv7 SBC (Atmel SAMA5D27-SOM with 128MB RAM).
The firmware update process requires a .tar.gz file to be uploaded via this interface, then untarred/unzipped and further processes run on it.
After successful upload using busboy module, I noticed that the memory usage is still high (see attached image) even though the upload is streamed directly into a file (code below).
Further processes implemented with the child_processes after the file upload are then killed due to the system being out of memory.
I have implemented the --max-old-size-space-size flag to = 32MB as i've read this sets a lower threshold for when the GC starts.
I have tried various methods of using child_processes APIs including: 1. execSync 2. async exec (as described in node.js docs) 3. spawn with {detached: true}
/* imports omitted for brevity */
module.exports = async (req, res, next) => { // eslint-disable-line no-unused-vars
try {
/* File validation omitted for brevity */
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
/* Further file validation omitted for brevity,
File transfer debugging omitted for brevity */
// Handle saving file stream to file
const saveTo = path.join(updatesDir, path.basename(filename));
file.pipe(fs.createWriteStream(saveTo));
});
busboy.on('finish', () => {
return res.json({ updateFilename });
});
// This is required stream req progress back to browser
return req.pipe(busboy);
} catch (error) {
console.error(error)
next(error);
}
}
I understand that node.js uses a garbage collector, however the memory taken for the file upload isn't freed for later processes to use.
Is there something wrong with how I am streaming the file upload to a file? Why is Node.js not releasing the memory? Is there a way to trigger GC manually in this instance?
