2

I've written a small BeautifulSoup-based scraper that retrieves portion of content from a multilingual site. Unfortunately if I try to scrape the US version (www.) I am redirected to the UK (uk.) site (where I'm based).

I was wondering if it was possible to "pretend" to be in the US, for example using a US-based IP or something similar.

As you can imagine from the level of the problem and terminology used, I'm not a professional programmer. Thanks

fghersi
  • 33
  • 6
  • http://jakeaustwick.me/python-web-scraping-resource/ – bhansa Jul 05 '16 at 16:04
  • can you share the link? It is probably down to setting a header or setting a cookie http://stackoverflow.com/questions/38066229/scrapy-formrequest-trying-to-send-a-post-request-formrequest-with-currency-c/38069653#38069653 – Padraic Cunningham Jul 05 '16 at 19:15
  • Sorry for the late reply! Unfortunately it's a project I am preparing for professional reasons so I can't share it. – fghersi Aug 24 '16 at 15:06

0 Answers0