Convert Response Variable to JSON file

Question

I'm trying to convert the page at https://lpl.qq.com/web201612/data/LOL_MATCH_DETAIL_7325.js to a json file so that I can parse it into a pandas table.

I'm using the python requests library and my function looks like this:

[1]def lpl_get_match_data(url, test):
[2]    good_url = "https://lpl.qq.com/web201612/data/LOL_MATCH_DETAIL_7325.js"
[3]    json_file = requests.get(good_url)
[4]    json_content = json_file.json()
[5]    if test == True:
[6]        pprint.pprint(json_content)
[7]    return(json_content)

When I run this function I get an error, "JSONDecodeError: Expecting value: line 1 column 1 (char 0)".

I did the obvious "google the error" and checked a bunch of the responses and it looked like in a lot of cases people had empty files or weren't getting responses. I tried changing line 3 to requests.get(good_url).content to confirm that I was getting the proper response and it looked like I was. I'm wondering if anyone could point me to why this page isn't readable as a json object.

score 2 · Accepted Answer · answered Jul 25 '21 at 19:02

2

The problem is that the page you are trying to read constains a Javascript object, instead of a JSON file. You can convert the object you are getting with demjson library, using: demjson.decode(json_file.content[12:-1]). Note that content is a string, and therefore you can slice the parts "var dataObj = {" and "}", with the given indexes. You can use this instead of your third line, and this should fix you problem.

answered Jul 25 '21 at 19:02

Stefan Radojevic

61
2

Worked like a charm. I figured the issue was that the page directed to a java object and not a json file, but had no idea how to fix that. Thank you so much. – S.Slusky Jul 25 '21 at 19:25
1

You're welcome! Enjoy and happy coding! :) – Stefan Radojevic Jul 25 '21 at 19:29

Convert Response Variable to JSON file

1 Answers1