Parsing mutual fund JSON data

Question

I am trying to pull out data from interactive chart and I have the JSON file from this link

http://www.tadawul.com.sa/Charts/MutualFundChartDataDownloader

I tried importing this to Python so I can extract the data, but I am facing errors

The code I wrote:

import urllib
import json

htmltext = urllib.urlopen("https://www.tadawul.com.sa/Charts/MutualFundChartDataDownloader?actionTarget=mutualFundChartData&mutualFundSymbol=006038&format=json")

data = json.load(htmltext)

print data["unitPrice"]

What I am really trying to import is the following data

"valuationDate"
"valudationDateAsDate"
"mutualFundNav"
"unitPrice"

I want to get these data one by one so I can copy it and use it in Excel.

All is what I am trying to do is to get the prices for this chart

I would suggest using the library [`requests`](http://docs.python-requests.org/en/master/) for fetching things form the web. It's a much better package. — kylieCatt, Aug 21 '16 at 08:20

101 · Answer 1 · 2016-08-18T01:49:08.237

Running your (slightly modified) code shows the website data you are actually getting.

import urllib

htmltext = urllib.urlopen("https://www.tadawul.com.sa/Charts/MutualFundChartDataDownloader?actionTarget=mutualFundChartData&mutualFundSymbol=006038&format=json")
print htmltext.read()

It's not what you expect, it returns a HTML doc with:

<html><head><title>Request Rejected</title></head><body ... </html>

Your request for that URL is being blocked when calling from Python as opposed to a browser. This code makes a request that includes a user agent, which allows the download to work:

import urllib2
import json

url = "https://www.tadawul.com.sa/Charts/MutualFundChartDataDownloader?actionTarget=mutualFundChartData&mutualFundSymbol=006038&format=json"
request = urllib2.Request(url, headers={'User-Agent' : ''})
result = urllib2.urlopen(request)
data = json.loads(result.read())

for entry in data:
    print entry[u'valuationDate']

score 0 · Answer 2 · answered Oct 27 '19 at 06:03

Use this code snippet:

import urllib.request as ur
import json

link= "https://www.tadawul.com.sa/Charts/MutualFundChartDataDownloader?actionTarget=mutualFundChartData&mutualFundSymbol=006038&format=json"

file = ur.urlopen(link)

# convert to json
file = json.load(file)

# parse as dataframe 
import pandas as pd
df= pd.DataFrame()
# get columns names
cols =list(file[0].keys())
# build your data frame
for c in cols: 
    df[c]= [f[c] for f in file]
# done

the final output of your data will look like :

then save your data as an excel sheet.

df.to_excel("output.xlsx")

Hope this helps.

score 0 · Answer 3 · answered Oct 27 '19 at 11:27

0

As @kylie.a suggested in a comment to the question itself, it's far easier to do it with requests:

import requests
import json
import pandas as pd

link = [your url above]
res = requests.get(link)
df = pd.read_json(res.text)

answered Oct 27 '19 at 11:27

Jack Fleeting

24,385
6
23
45

Parsing mutual fund JSON data

3 Answers3