1

I'm scraping data from a website since 3 months ago, but today I can't access the website anymore, neither with my web-browser. The site is still accessible via mobile phone. I have this message when I test a link in Scrapy shell : 

twisted.internet.error.TCPTimedOutError: TCP connection timed out: 10060

Do you think I'm blacklisted? Or is it a technical problem?

I'm using python3 + scrapy + splash via docker all under Windows 10.

Adrian Mole
  • 49,934
  • 160
  • 51
  • 83
Frederic
  • 115
  • 10

1 Answers1

2

I think my IP was banned because I can't access the website via my web-browser or even on other computer with the same connection. And the website works fine when using another IP address. I fixed this by adding Scrapoxy in my project.

Frederic
  • 115
  • 10
  • You can avoid this behavior in the future by some other approaches like increasing up they delay in your request and modifying the number of concurrent requests. Before using proxies or some other utilities it is better to ensure basic checks – Ahmed Buksh Jan 17 '20 at 13:43
  • If my IP is banned, what will "increasing up they delay" or "modifying the number of concurrent requests" exactly change ? – Frederic May 07 '20 at 00:18