4

I would like interact with Google Trends using Python Requests module. Google Trends requires authentication in order to access Google Trends data. Is there a way to programmatically log in with Requests? It does not seem like OAuth 2.0 will work for me since I am not using Google API.

Kara
  • 6,115
  • 16
  • 50
  • 57

1 Answers1

1

I think you actually get more interesting data by grabbing the raw JSON that it uses to build the graphs. It includes the related headlines that don't come with the CSV download. This works for a few queries (5?) before you reach the quota.

import re
import requests

_GOOGLE_TRENDS_URL = 'http://www.google.com/trends/trendsReport?hl=en-US&content=1&q=%s&hl=en-US&content=1'

term = 'foo'

response = requests.get(_GOOGLE_TRENDS_URL % term)

if response.status_code == requests.codes.ok:
    data_line = [l for l in response.content.splitlines() if 'var chartData' in l][0]
    chart_data = re.sub(r'.*var chartData = (.*?);.*', r'\1', data_line)
    # Fix for date representation
    chart_data = re.sub(r'new Date\((\d+), (\d+), (\d+)\)', r'"\1-\2-\3"', chart_data)

    data = json.loads(chart_data)

#data = {
# ...,
#    "rows": [
#       [
#        {
#         "f": "January 2004",
#         "v": "2004-0-16"
#        },       # Date
#        null,    # annotation
#        null,    # annotation text
#        91,      # count
#        null,    # annotation (2?)
#        null,    # annotationText (2?)
#        true     # certainty
#       ],
#...
#   ]
#}

    for row in data['rows']:
        if '2013' in row[0]['v']:
            print '%s: %d' % (row[0]['f'], row[3])
else:
    print response.status_code
    print response.text

Yields:

January 2013: 21
February 2013: 21
March 2013: 21
April 2013: 20
May 2013: 20
June 2013: 20
July 2013: 20
August 2013: 21
September 2013: 19
October 2013: 20
November 2013: 21
December 2013 (partial data): 22
brechin
  • 569
  • 4
  • 7