I need to send a lot of requests to a server whose flow limits are unclear. To combat this, I decided to send requests repeatedly until I receive a successful status - using he following code:
function sendRequest(url, body, isGood) {
return new Promise((resolve, reject) => fetch(url, body).then(
resp => {
if(isGood(resp)){
resolve(resp);
} else {
reject(resp.status)
}
}).catch(err => reject(err)))
}
function recursionBypass(func, ...args){
return func(...args)
}
function requestUntilSucceed(url, body, isGood, name, attempt=1) {
return new Promise((resolve, reject) => {
sendRequest(url, body, isGood)
.catch((status)=>{
recursionBypass(requestUntilSucceed, url, body, isGood, name, attempt+1)})
.then((resp) => resolve(attempt))
})
}
Without including details about the server in question, I wrote a quick test for this (create & wait for 500 requests):
function () {
let promises = [];
for(let i=0; i<300; i++) {
promises.push(new Promise((resolve,reject) =>
{createRequest(requestArg, i)
.then((attempt) => {console.log("Request: ", i, "succeeded on attempt: " + attempt); resolve(attempt)})
}
)
)
};
return Promise.all(promises)
})
await function()
Assume:
requestArg
: a global variable (the same arg is used for all requests for this test)createRequest
: a function that creates the url, body and isGood callback then callssendRequest
that builds the url and body based onrequestArg
and returns as follows:return requestUntilSucceed(url, body, isGood, name)
I've observed that about 200 requests are accepted immediately and the others keep retrying. However, at some point, the program exits without finishing all requests successfully. Since there is no mechanism in place to limit the number of tries explicitly, I'm wondering how this can happen. I suspected it had to do with the limit on recursion depth so I started using recursionBypass
as above so the interpreter can't tell the function is calling itself, but this didn't make a difference. Any ideas as to why it would exit early ? I know the server accepts requests until some count, then rejects them for a cooldown period of ~1min before beginning to accept again. The rate of acceptance after each cooldown period is ambiguous which is why I can't write a rule-based throttler. The issue is that my program exists (no error printed to console) before the minute is up. A lot of attempts are made in that minute so maybe I'm still surpassing some kind of javascript limit I'm not familiar with ?