0

If a website states to not crawl their site, is it still legal to use a tool like Yahoo Pipes or YQL to create mash ups?

I don't see how it is any different than direct web scraping except a third party is retrieving the data.

Thanks.

Scott Wegner
  • 7,263
  • 2
  • 39
  • 55
George Richardson
  • 53
  • 1
  • 6

1 Answers1

0

Yes, you are right. If a site says not to crawl data from their site, that means its violation of their terms whichever way you scrape data from the site. For eg. from LinkedIn's terms and conditions (http://www.linkedin.com/legal/user-agreement) in point 23 in "Don’t undertake the following" -

Use manual or automated software, devices, scripts robots, other means or processes to access, “scrape,” “crawl” or “spider” any web pages or other services contained in the site;

which clearly states "manual or automated software, devices, scripts robots, other means", and thats cover any possible way to scrape.

theharshest
  • 7,767
  • 11
  • 41
  • 51