0

Is it possible to kill/stop multi curl requests if one of the requests receives the status code 429?

I've been trying to find a solution to this as I pull data from an API and need to avoid going over the rate limit.

These requests are asynchronous so I'm unsure how I would do this.

Joel
  • 385
  • 1
  • 4
  • 13

1 Answers1

1

well, you should be able to cancel them at will with a CURLOPT_PROGRESSFUNCTION, have a global variable for wether or not to cancel transfers, a function that import it (with the global $var syntax), and make it return 1 when its time to cancel, eg $abort=false;ecurl_setopt($ch,CURLOPT_PROGRESSFUNCTION,function($a,$b,$c,$d,$e){global $abort;return (int)!$abort;}); - then just make $abort=true; when its time to abort them. that said, you can use CURLOPT_MAX_RECV_SPEED_LARGE to limit the speed of the transfers, if its a speed rate limit you're exceeding

edit: note that you also need to set CURLOPT_NOPROGRESS to false for CURLOPT_PROGRESSFUNCTION to be called at all.

hanshenrik
  • 19,904
  • 4
  • 43
  • 89
  • Thanks, seems to stop most requests but not all, better than nothing I suppose. Instead of using the abort variable I use memcache to store the variable and check it that way. Hopefully will stop me from getting too many 429 responses, a bit difficult to implement a rate limiter with async requests. – Joel Aug 03 '17 at 22:20