I am trying to download about 800MB per file. This is the code to achieve it:
function makeRequest(url,name,callback){
name = promObj.Name;
requestsdone =-1;
request(url,{auth:auth}, function (err,respnse,body) {
})
.on('error', function (err) {
console.log(err)
})
.on('end', function () {
requestsdone++;
}).pipe(fs.createWriteStream('./app/data/' + name[requestsdone] + '.zip'));
callback();
}
async.eachLimit(promObj.requestURLS, 2, makeRequest, function(err) {
if(err) reject(promObj)
else{
resolve(promObj);
}
});
I updated Node.js to version 9.5.0 and get this error now:
<--- Last few GCs --->
[12060:000001FED372E610] 487169 ms: Mark-sweep 28.7 (35.4) -> 28.7 (35.4)
MB, 33.6 / 0.0 ms allocation failure GC in old space requested
[12060:000001FED372E610] 487205 ms: Mark-sweep 28.7 (35.4) -> 28.7 (35.4)
MB, 36.2 / 0.0 ms last resort GC in old space requested
[12060:000001FED372E610] 487240 ms: Mark-sweep 28.7 (35.4) -> 28.7 (35.4)
MB, 34.7 / 0.0 ms last resort GC in old space requested
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 000001311C6254D9 <JSObject>
1: stringSlice(aka stringSlice) [buffer.js:586] [bytecode=0000031E27B6DEE1 offset=11](this=000003FACAA022D1 <undefined>,buf=000000CDF7528259 <Uint8Array map = 00000392BE4C4D01>,encoding=000003FACAA022D1 <undefined>,start=0,end=801130252)
2: toString [buffer.js:664] [bytecode=0000031E27B6DB29 offset=148](this=000000CDF7528259 <Uint8Array map = 00000392BE4C4D01>,encoding=000003FACAA022D1...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
How could I solve this? Thanks in advance.
PS: This question refers to Requests with request-promise pipe result (Large data) ,which I asked some time ago. I restructured my code and thought that this problem is worth a new, sepperated question because the original topic of the other question was a different one.
