I'm currently grabbing only the first page of google results for a query, but I want to grab the first 5 pages.
gets a string like: https://encrypted.google.com/search?hl=en&q=site%3Asomedomain.com&start=0
the variable urls
gets all 10 results for the first page, but I started adding conditions to check for 10 urls on this first page, if that is true and there are 10 urls, I want it to keep going to the next url e.g. (provided the next url has 10 results as well) using something like follow_link()
and urls below :
https://encrypted.google.com/search?hl=en&q=site%3Asomedomain.com&start=10
https://encrypted.google.com/search?hl=en&q=site%3Asomedomain.com&start=20
https://encrypted.google.com/search?hl=en&q=site%3Asomedomain.com&start=30
https://encrypted.google.com/search?hl=en&q=site%3Asomedomain.com&start=40
https://encrypted.google.com/search?hl=en&q=site%3Asomedomain.com&start=50
How do I go about doing this? Could anyone please help me out?