Stack Overflow
Stack Overflow

Questions tagged [scraperwiki]

ScraperWiki was an online tool for Screen Scraping.

ScraperWiki ScraperWiki was a platform for writing and scheduling screen scrapers, and for storing the data they generate. It support Ruby, Python and PHP. A later version of the service was called QuickCode, which has also been decommissioned.

"Scraper" refers to screen scrapers, programs that extract data from websites. "Wiki" means that any user with programming experience can create or edit such programs for extracting new data, or for analyzing existing datasets.

68 questions
0
votes
1 answer

Why does this ScraperWiki for an ASPX site return only the same page of search results?

I'm trying to scrape an ASP-powered site using ScraperWiki's tools. I want to grab a list of BBSes in a particular area code from the BBSmates.com website. The site displays 20 BBS search results at a time, so I will have to do form submits in order…
Kirkman14
  • 1,506
  • 4
  • 16
  • 30
0
votes
1 answer

ScraperWiki: simple_html_dom library

I would like to know if the simple_html_dom library from ScraperWiki: require 'scraperwiki/simple_html_dom.php'; Is the same or uses the same methods as this one: PHP Simple HTML DOM Parser I want to know this because it would be an easy way to…
rfc1484
  • 9,441
  • 16
  • 72
  • 123
0
votes
1 answer

ScraperWiki: How to save html so it only gets loaded once

When I execute a scraper, it loads the url using this method: $html = scraperWiki::scrape("foo.html"); So every time I add new code to the scraper and want to try it, it loads again the html, which takes a fair amount of time. Is there anyway…
rfc1484
  • 9,441
  • 16
  • 72
  • 123
0
votes
2 answers

Scraperwiki scrape query: using lxml to extract links

I suspect this is a trivial query but hope someone can help me with a query I've got using lxml in a scraper I'm trying to build. https://scraperwiki.com/scrapers/thisisscraper/ I'm working line-by-line through the tutorial 3 and have got so far…
elksie5000
  • 7,084
  • 12
  • 57
  • 87
0
votes
1 answer

Saving and Resuming on scraperwiki - CPU time

This is my first time doing this, so I better apologize in advance for my rookie mistakes. I'm trying to scrape legacy.com for the first page results from searching for a first and last name within the state. I'm new to programming, and was using…
Jon P
  • 1
0
votes
1 answer

ScraperWiki/Python: filtering out records when property is false

I'm using the following code on ScraperWiki to search Twitter for a specific hashtag. It's working great and is picking out any postcode provided in the tweet (or returning false if none is available). This is achieved with the line…
Martin
  • 117
  • 2
  • 10
-1
votes
3 answers

Access data of a wiki scraper and store it in a local database

Hi i have written a scraper on scraper-wiki which scrape a web page and store the data in scraper-wiki database.Now i want to write a program in python which will go to scraper-wiki API and fetch the data and stores in a sq lite database of local…
Binit Singh
  • 973
  • 4
  • 14
  • 35
-1
votes
1 answer

Scraping data from multiple links within a site

I would like to use scraperwiki and python to build a scraper that will scrape large amounts of information off of different sites. I am wondering if it is possible to point to a single URL and then scrape the data off of each of the links within…
Cetus
  • 31
  • 6
1 2 3 4
5