3

I would like to scrape a site making requests using different IPs in order not to be blocked as it happens when all requests are coming from one IP address only. The site in question blocks me after a while even if I pause 20-30 seconds between each request.

It has a very restrictive throttle limit and I will need to use many proxies to overcome that. It's easy to put a list of proxy IPs in an array and cycle through that, but I miss the fundamental feature of changing the proxy configuration at runtime programmatically in Ruby Watir.

0 Answers0