-2

I need to scrape google for the first 10-20 URL results for a keyword, I found a lib called pygoogle but when I run it I get a syntax error: here is my code:

from pygoogle import pygoogle
g = pygoogle('quake 3 arena')
g.pages = 5
print '*Found %s results*'%(g.get_result_count())
g.get_urls()

Alright, it seems like pygoogle does not work anymore because Google deprecated the SOAP interface- does anyone know an alternative?

Ken Dotx
  • 1
  • 1
  • 3
    What is the syntax error do you get? – marsh Nov 05 '14 at 19:25
  • Are you using Python 2 or Python 3? That print syntax only works with the former. – Kevin Nov 05 '14 at 19:26
  • If you need a alternative to html scraping because pypi does not work I used this tutorial. It is very fast and easy to get started: http://docs.python-guide.org/en/latest/scenarios/scrape/ – marsh Nov 05 '14 at 19:29
  • Note that `(g.get_result_count())` is not a 1-element tuple, it's just the single value wrapped in unnecessary and meaningless parentheses. Commas create tuples, not parentheses. (This happens to work anyway because of a quirk of `%`-formatting, but it's misleading—if you want to rely on that quirk, leave the parens off.) – abarnert Nov 05 '14 at 19:29
  • I use Python 3 and I get "Invalid Syntax". But as TkTech said it seems like pygoogle does not work anymore. – Ken Dotx Nov 05 '14 at 19:30
  • OK, so you knew the parens were required and just forgot them. This seems like a simple typo that isn't worth having an answer. If you have a question about how to use Google's APIs, that's a completely separate question, and shouldn't be shoehorned into this one. (It may be a question that already has a dup, or that isn't appropriate for SO, or whatever, but it's definitely not part of this question.) – abarnert Nov 05 '14 at 19:39
  • However, as a quick hint: Google now has a huge variety of different APIs. All of them can be accessed by relatively simple JSON REST calls that don't really need a custom library, but most of them do have an official custom library from Google anyway, usually with Python bindings. Do a bit of research on the API you want to use before asking about it, because a general "how do I call Google from Python" isn't going to have any useful answers. – abarnert Nov 05 '14 at 19:41

1 Answers1

2
  1. If you're using the pygoogle from pypi, this package no longer works since Google deprecated the SOAP interface. Even if you fix the syntax error, this will not work.
  2. Always include the error message/stack trace when you post a question!
  3. You are probably using python 3, where print is a function (print('Hello!')) and not a keyword (print 'Hello!') by default. Additionally, old style % formatting is deprecated, use .format() instead.
TkTech
  • 4,729
  • 1
  • 24
  • 32
  • 1
    `%`-formatting is not deprecated. There was a bit of a war over it, and the final conclusion was that both have their place in different cases, `{}`-formatting should be encouraged as the go-to choice when you don't have a compelling reason otherwise, but neither should be deprecated. – abarnert Nov 05 '14 at 19:28
  • Thanks for the correction @abarnert, I wasn't aware that it had been settled. – TkTech Nov 05 '14 at 19:32
  • I use python 3.4.2 so I forgot the parantheses for the print function. Since pygoogle does not work anymore does anyone know an alternative? – Ken Dotx Nov 05 '14 at 19:36