Stack Overflow
Stack Overflow

Questions tagged [json-normalize]

This tag is specifically for questions using pandas.json_normalize to flatten JSONs, dicts and lists of dicts

228 questions
361
votes
13 answers

Split / Explode a column of dictionaries into separate columns with pandas

I have data saved in a postgreSQL database. I am querying this data using Python2.7 and turning it into a Pandas DataFrame. However, the last column of this dataframe has a dictionary of values inside it. The DataFrame df looks like this: Station ID…
llaffin
  • 3,651
  • 3
  • 10
  • 6
312
votes
27 answers

How can I convert JSON to CSV?

I have a JSON file I want to convert to a CSV file. How can I do this with Python? I tried: import json import csv f = open('data.json') data = json.load(f) f.close() f = open('data.csv') csv_file = csv.writer(f) for item in data: …
little_fish
  • 4,169
  • 5
  • 20
  • 8
62
votes
4 answers

Pandas read nested json

I am curious how I can use pandas to read nested json of the following structure: { "number": "", "date": "01.10.2016", "name": "R 3932", "locations": [ { "depTimeDiffMin": "0", "name": "Spital am…
Georg Heiler
  • 16,916
  • 36
  • 162
  • 292
49
votes
3 answers

pandas json_normalize with very nested json

I have been trying to normalize a very nested json file I will later analyze. What I am struggling with is how to go more than one level deep to normalize. I went through the pandas.io.json.json_normalize documentation, since it does exactly what I…
Daniel Vargas
  • 980
  • 2
  • 13
  • 21
19
votes
2 answers

flattening nested Json in pandas data frame

I am trying to load the json file to pandas data frame. I found that there were some nested json. Below is the sample json: {'events': [{'id': 142896214, 'playerId': 37831, 'teamId': 3157, 'matchId': 2214569, 'matchPeriod': '1H', …
Zephyr
  • 1,332
  • 2
  • 13
  • 31
17
votes
2 answers

How to normalize json correctly by Python Pandas

I want to do is load a json file of forex historical price data by Pandas and do statistic with the data. I have go through many topics on Pandas and parsing json file. I want to pass a json file with extra value and nested list to a pandas…
chris198725
  • 173
  • 1
  • 1
  • 5
10
votes
4 answers

How to download a nested JSON into a pandas dataframe

Looking to sharpen my data science skills. I am practicing url data pulls from a sports site and the json file has multiple nested dictionaries. I would like to be able to pull this data to map my own custom form of the leaderboard in matplotlib,…
Hokemon
  • 111
  • 5
7
votes
2 answers

How to json_normalize a column in pandas with empty lists, without losing records

I am using pd.json_normalize to flatten the "sections" field in this data into rows. It works fine except for rows where the "sections" is an empty list. This ID gets completely ignored and is missing from the final flattened dataframe. I need to…
ldacey
  • 518
  • 8
  • 16
6
votes
3 answers

How to update a pandas dataframe, from multiple API calls

I need to do a python script to Read a csv file with the columns (person_id, name, flag). The file has 3000 rows. Based on the person_id from the csv file, I need to call a URL passing the person_id to do a…
Malkath
  • 77
  • 3
  • 13
5
votes
3 answers

Why does pandas.json_normalize(json_results) raise a NotImplementedError?

I have a json variable named json_results and I am running pandas.json_normalize(json_results). It raises the following error: in _json_normalize raise NotImplementedError NotImplementedError How can I resolve this?
pwlt1998
  • 51
  • 1
  • 2
5
votes
1 answer

How to json_normalize a column with NaNs

This question is specific to columns of data in a pandas.DataFrame This question depends on if the values in the columns are str, dict, or list type. This question addresses dealing with the NaN values, when df.dropna().reset_index(drop=True) isn't…
Trenton McKinney
  • 56,955
  • 33
  • 144
  • 158
5
votes
1 answer

Nested JSON Array to Python Pandas DataFrame

I'm trying to expand nested json array in pandas dataframe. That's the JSON I have: [ { "id": "0001", "name": "Stiven", "location": [{ "country": "Colombia", "department": "Chocó", …
Stiven Ramírez Arango
  • 420
  • 4
  • 14
4
votes
1 answer

How do I json_normalize() a specific field within a df and keep the other columns?

So here's my simple example (the json field in my actual dataset is very nested so I'm unpacking things one level at a time). I need to keep certain columns on the dataset post…
Anonymous
  • 453
  • 1
  • 6
  • 14
4
votes
3 answers

How to create pandas dataframe from nested json with dictionary

I'm trying to create a pandas dataframe form json file. I've seen a multiple solutions to this problem which uses built in functions from_dict/json_normalize yet I'm unable to apply it to my code. Here's how my data is structured in json file: …
Towelie
  • 43
  • 4
4
votes
1 answer

How to split a pandas column with a list of dicts into separate columns for each key

I'm analyzing Political Advertisements from Facebook, which is a dataset released here, by ProPublica. Here's what I mean. I have an entire column of targets that I want to analyze but it's formatted in a very inaccessible way for someone of my…
Jeffrey Lin
  • 53
  • 1
  • 7
1
2 3
…
15 16