Python and JSON(very long file)

Question

I've searched on forum, I have seen a lot of discussions about it but nothing was so specific. So I have a very long json file and I want to load it in python as dictionary, not list. You can see the file here https://www.bicing.cat/availability_map/getJsonObject

This is my code to load file but everytime I was running it, I received error : TypeError: list indices must be integers, not str

import json
import requests

if __name__=='__main__':
    response = requests.get("https://www.bicing.cat/availability_map  /getJsonObject")
    data=response.json()


    print data["typeStation"][22]

`TypeError: list indices must be integers, not str`: well, your `data` is a `list`, `"typeStation"` is a `str`. Does that tell you anything? — Norrius, Feb 18 '18 at 16:27
Yeah, I know that `data` is a `list`.... How can I convert `data` in `dict` to perform searching inside json file? — Francesco Alotto, Feb 18 '18 at 16:30
Please add a sample expected output to your question. I don't really understand how you want to make a dict out of a list. — Norrius, Feb 18 '18 at 16:38
Are you sure you don't need something like this: `print([item for item in data if item['id'] == '22'])` — Anton vBR, Feb 18 '18 at 16:45
I want for example a function allowing to order the bike-stations by available “slots” and display first N stations. The user can also choose to have the results in ascending or descending order (by default descending). The parameters are: -N: number of stations to display. It is optional and by default N=10. -order: It is optional and the default value is descending To perform this I have tried to read json file as dictionary, but the function `json.load` or `json.loads` returns list. — Francesco Alotto, Feb 18 '18 at 16:45
@FrancescoAlotto Ok, why didn't you say so from start.. hehe — Anton vBR, Feb 18 '18 at 16:47
@AntonvBR because I want to solve the loading problem... :) sorry for my bad english !! — Francesco Alotto, Feb 18 '18 at 16:52
@FrancescoAlotto This is basically done with pandas in 2 rows. See my example below. — Anton vBR, Feb 18 '18 at 17:07
@AntonvBR thank you so much!!!! GEOJSON is a very interesting function. — Francesco Alotto, Feb 18 '18 at 19:19

Anton vBR · Accepted Answer · 2018-02-18T19:28:13.660

Use pandas library to handle table-like data:

import pandas as pd

# Read to dataframe
df = pd.read_json("https://www.bicing.cat/availability_map/getJsonObject")

def return_stations(df, n=10, ascending=False):
    """a function allowing to order the bike-stations by available “slots” 
       and display first N stations. The user can also choose to have the 
       results in ascending or descending order (by default descending). The 
       parameters are: -N: number of stations to display. It is optional and 
       by default N=10. -order: It is optional and the default value is 
       descending"""
    return (df.sort_values(by='slots',ascending=ascending)
            .head(n).set_index('id').to_dict('i'))

# Let's print the first 3 in descending order
print(return_stations(df, n=3, ascending=False))

Returns the following:

{10: {'address': 'Carrer Comerç',
  'addressNumber': '27',
  'bikes': 0,
  'district': 1,
  'lat': 41.38498,
  'lon': 2.18417,
  'name': '10 - C/ COMERÇ, 27',
  'nearbyStations': '9,14,115,389',
  'slots': 33,
  'stationType': 'BIKE',
  'status': 'CLS',
  'zip': 8003},
 213: {'address': 'Sant Fe de Nou Mèxic',
  'addressNumber': '2',
  'bikes': 0,
  'district': 6,
  'lat': 41.393783,
  'lon': 2.135078,
  'name': '213 - C/ SANTA FE DE NOU MÈXIC, 2',
  'nearbyStations': '207,208,214,215',
  'slots': 33,
  'stationType': 'BIKE',
  'status': 'OPN',
  'zip': 8017},
 326: {'address': 'Balmes',
  'addressNumber': '409',
  'bikes': 0,
  'district': 6,
  'lat': 41.407384,
  'lon': 2.1383,
  'name': '326 - C/BALMES, 409',
  'nearbyStations': '321,327,328,330',
  'slots': 33,
  'stationType': 'BIKE',
  'status': 'OPN',
  'zip': 8022}}

Ohh and another thing... saw you got lat lng there: We could write a geojson file and plot it on map dropping the file on geojson.io. Here is a function for that:

def return_geojson(df, latlng):
    d = dict(type="FeatureCollection",features=[])

    for ind,row in df.fillna('').iterrows():
        p = row.drop(latlng).to_dict()
        g = dict(type="Point",coordinates=list(row[latlng])[::-1])
        f = dict(type="Feature", properties=p, geometry=g)
        d['features'].append(f)

    return d

with open('map.geojson','w') as f:
    d = return_geojson(df,['lat','lon'])
    json.dump(d,f)

Result:

Python and JSON(very long file)

1 Answers1