DO NOT USE THIS TAG. It is under an active cleanup: https://meta.stackoverflow.com/q/305314 Use [web-scraping] if your question is about scraping information from web resources (there is also [screen-scraping]) or use [pdf-scraping] if your question is about scraping information from pdf files. Use [data-extraction] if you need to extract data from other resources.
Questions tagged [scrape]
1204 questions
-2
votes
1 answer
Scrape Javascript page with Python or other
I want to scrape the team grids for each game on the following website:
http://mc.championdata.com/nrl/
and I believe the code below is for away teams:
brad58
- 1
-2
votes
1 answer
Can php save scraped data into a cache file with a file name as CURRENT DATE?
I'm asking because I'd like to have a graph with historical data, including from the most updated date. I know how to save scraped data as a certain name, and have it saved into a cache folder.
If it's possible instead of saying...save current file…
Masteryogurt
- 199
- 4
- 14
-2
votes
1 answer
How can I scrape data from a function which includes loops
I'm learning Web Scraping (with Python and Beautiful Soup) and I encountered with a problem on how to scrape data within a function which contains loops. The data I'm trying to get is within the condition of if,else statement as shown below.(Page…
Sreekant Shenoy
- 1,420
- 14
- 23
-2
votes
1 answer
Is Python a suitable tool for automating data scraping?
I am working on a project which involves working with a large amount of data. Essentially, there exists a large repository on some website of excel files that can be downloaded. The site has several different lists of filters and I have several…
niccalis
- 134
- 1
- 7
-2
votes
1 answer
Where can I find the list of web scraping projects to practice?
I am looking to practice the web scraping projects. Can anyone tell me where can I find such list or give me some examples of web scraping projects?
nitinvijay23
- 1,781
- 3
- 13
- 11
-2
votes
1 answer
Vb.Net Scraped Data MsgBox Shows Line Breaks but RichTextBox Does Not
So I'm scraping data and I've got my xpath parsing out the line I need.
I go line by line and dump out the data so I know I'm looking in the right place and for every listing it's showing several lines which is good. Exactly what I expected.
Example…
Max Better
- 61
- 10
-2
votes
1 answer
scrape the data-title and data-video-id on single page
I want to scrape the data-title and data-video-id but i don't know how to write the script with imacros
The Photos
Dipta Harimbawa
- 3
- 1
-2
votes
2 answers
writing a list with multiple data to a csv file in separate columns in python
import csv
from itertools import izip
if l > 0:
for i in range(0,l):
combined.append(str(questionList[i]).encode('utf-8') + str(viewList[i]).encode('utf-8'))
# viewcsv.append(str(viewList[i]).encode('utf-8'))
# …
Rohit Raj
- 1
- 1
- 2
-2
votes
1 answer
undefined method `[]' for nil:NilClass while using Nokogiri
I am using Nokogiri to scrape data from a HTML document, but I'm running into the following error:
`block in ': undefined method `[]' for nil:NilClass (NoMethodError)
This is the code to reproduce the problem:
require 'rubygems'
require…
Anshul Kalra
- 198
- 3
- 13
-2
votes
2 answers
Scrape data from HTML
This is the page I'm trying to scrape from, http://www.footballlocks.com/nfl_point_spreads_week_1.shtml , I want to end up with a simple data.frame with 4 columns so I can perform further analysis. I've have tried using the XML package but with not…
RcodeNFL
- 9
- 1
- 1
- 1
-2
votes
2 answers
Scrape content from a webpage
I'm trying to scrap content from this webpage:
www.motorcyclemonster.com/motorcycle-events.html, the code I'm using is:
TrevTech
- 49
- 6
-2
votes
1 answer
Scrape urls from google's top page in python
I need to scrape google for the first 10-20 URL results for a keyword, I found a lib called pygoogle but when I run it I get a syntax error: here is my code:
from pygoogle import pygoogle
g = pygoogle('quake 3 arena')
g.pages = 5
print '*Found %s…
Ken Dotx
- 1
- 1
-2
votes
1 answer
How to scrape all data from website with Jsoup?
I need to scrape all data from those sites:
http://outfit7.com/privacy-policy/
http://www.gameloft.com/privacy-notice/
I use JSOUP. And program must scrape all text from site. As you see those sites have different structures. So I should use…
user3661720
- 27
- 12
-2
votes
1 answer
Combining string (URL) with Array
I am trying to scrape some data. B is the number of pages required for 1000 items to be scraped - there are 22 per page.
usernum = 1000
b = usernum.to_i/22
Array.new(b) {|b| b+1}
I have an array for the number of pages (1000/22 = 45 int.) -…
user2208607
- 35
- 2
- 5
-2
votes
1 answer
How to scrape relative images
If i look at Amazon Button to add items to lists on their site - you can see it here:
http://www.amazon.co.uk/wishlist/get-button
How does it work? I'm pretty sure it scrapes the page somehow but it seems to get every image whether its a flash…
StudioTime
- 22,603
- 38
- 120
- 207