9

I am requesting youtube search terms for use with jquery autocomplete, but am having a hard time converting the URL response into a proper format.

In my (Django/Python) view I do:

data2 = urllib2.urlopen('http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&jsonp=window.yt.www.suggest.handleResponse&q=jum&cp=3')

(I hardcoded the search term = 'jump' for simplicity)

If I do data2.read() I get what I believe is JSON (copy-pasting the url into a browser also returns this.)

window.yt.www.suggest.handleResponse(["jum",[["jumpstyle","","0"],["jump","","1"],["jump around","","2"],["jump on it","","3"],["jumper","","4"],["jump around house of pain","","5"],["jumper third eye blind","","6"],["jumbafund","","7"],["jump then fall taylor swift","","8"],["jumpstyle music","","9"]],"","","","","",{}])

I need to return this in a format that jquery autocomplete can read. I know it will work if I can get it into a list, for example, mylist = ['jumpstyle', 'jump', 'jump around', ...]

and then convert it back into json before returning it:

json.dumps(mylist)

(This works if I directly define mylist directly as written above.)

But I cannot get from the data that is returned by the URL to either a simple list (which I then convert back into JSON) or to some form of JSON that I can return directly to be used by auto complete.

I've tried, among other things,

j2 = json.loads(data2)

and 

j2 = json.loads(data2.read())

Hope someone can help!

Ivo Wetzel
  • 46,459
  • 16
  • 98
  • 112
dkgirl
  • 4,489
  • 7
  • 24
  • 26

4 Answers4

14

remove the &jsonp=window.yt.www.suggest.handleResponse part

import json
import urllib2

data = urllib2.urlopen('http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&q=jum&cp=3')

j = json.load(data)
k = [i for i, j, k in j[1]]
l = json.dumps(k)
Xavier Barbosa
  • 3,919
  • 1
  • 20
  • 18
3

You are doing a JSON-P request which automatically wraps the JSON in a javascript callback function, the one you have specified in the request in fact :)

Strip away the JSON-P parameter from your request and you will get straight JSON directly from the request without having to do any extra python stuff at all.

This should be your request:

http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&q=jum&cp=3

and it will return:

["jum",[["jumpstyle","","0"],["jump","","1"],["jump around","","2"],["jump on it","","3"],["jumper","","4"],["jump around house of pain","","5"],["jumper third eye blind","","6"],["jumbafund","","7"],["jump then fall taylor swift","","8"],["jumpstyle music","","9"]],"","","","","",{}]
Jon Nylander
  • 8,743
  • 5
  • 34
  • 45
  • Ah... How do I get that to look like: ["jumpstyle", "jump", "jump around"...] I'm not sure how to manipulate these kinds of data: what IS it? A list, a string, a json object? I tried json.loads(returned_by_url) but get an error. – dkgirl Jan 08 '11 at 14:41
  • Xavier seems to have covered that :) – Jon Nylander Jan 08 '11 at 16:14
0

it's not json it's javascript, if you want to use it as json you must strip the javascript part:

j2 = json.loads(data2[37:-1])

but you can just change the url(remove the 'jsonp=window.yt.www.suggest.handleResponse' part) to have pure json output:

>>> data2 = urllib2.urlopen('http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&q=jum&cp=3')
>>> json.loads(data2.read())
[u'jum', [[u'jumpstyle', '', u'0'], [u'jump', '', u'1'], [u'jump around', '', u'2'], [u'jump on it', '', u'3'], [u'jumper', '', u'4'], [u'jump around house of pain', '', u'5'], [u'jumper third eye blind', '', u'6'], [u'jumbafund', '', u'7'], [u'jump then fall taylor swift', '', u'8'], [u'jumpstyle music', '', u'9']], '', '', '', '', '', {}]
virhilo
  • 6,568
  • 2
  • 29
  • 26
0

The output from the page is not a proper json encoded data. You need to remove the js function call wrapping it.

do this:

import urllib2
import re
import json

data2 = urllib2.urlopen('http://suggestqueries.google.com/complete/search?' +    
   'hl=en&ds=yt&client=youtube&hjson=t&jsonp=window.yt.' + 
   'www.suggest.handleResponse&q=jum&cp=3')

data = re.compile('^[^\(]+\(|\)$').sub('', data2.read())
parsedData = json.loads(data)

parsedData is python array now.

Mihai Toader
  • 12,041
  • 1
  • 29
  • 33