Normalize Monday.com API JSON output in Python

Question

I am ultimately trying to generate a csv from the JSON output of a Monday.com API request.

Below is my code so far. I am having trouble trying to flatten the JSON into a table.

import requests
import json
import pandas as pd

apiKey = "API Key Here"
apiUrl = "https://api.monday.com/v2"
headers = {"Authorization" : apiKey}

query2 = '{boards(ids:123456) {items{name, column_values{title text } } } }'
data = {'query' : query2}

json_data = json.loads(requests.post(url=apiUrl, json=data, headers=headers).text)

norm=pd.json_normalize(json_data, 'items',['data', 'boards'])

JSON Output from the API. I added some linebreaks for human readability.

{'data':
  {'boards':
  [{'items':
  [{'name': 'Item 1', 'column_values': [{'title': 'Person', 'text': 'Mark McCoy'}, {'title': 'Status', 'text': None}, {'title': 'Date', 'text': '2021-02-05'}]},
  {'name': 'This is a new item', 'column_values': [{'title': 'Person', 'text': ''}, {'title': 'Status', 'text': None}, {'title': 'Date', 'text': '2021-04-17'}]},
  {'name': 'Item 5', 'column_values': [{'title': 'Person', 'text': ''}, {'title': 'Status', 'text': None}, {'title': 'Date', 'text': '2021-02-13'}]},
  {'name': 'Item 2', 'column_values': [{'title': 'Person', 'text': ''}, {'title': 'Status', 'text': 'Done'}, {'title': 'Date', 'text': '2021-05-14'}]}]}]},
  'account_id': 00000000}

When I run the ,py file in terminal I get the below output

Traceback (most recent call last):
  File "/Users/markamccoy/Desktop/MondayPy/stack.py", line 14, in <module>
    norm=pd.json_normalize(json_data, 'items',['data', 'boards'])
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 336, in _json_normalize
    _recursive_extract(data, record_path, {}, level=0)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 309, in _recursive_extract
    recs = _pull_records(obj, path[0])
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 248, in _pull_records
    result = _pull_field(js, spec)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 239, in _pull_field
    result = result[spec]
KeyError: 'items'

I am total python noob and reading the pandas documentation has gotten me this far but I am a little out of my element.

score 1 · Accepted Answer · answered Jun 22 '21 at 01:05

If you use normalize.the result will looks like this.

df = pd.json_normalize(json_data['data']['boards'][0]['items'],record_path='column_values',meta=['name'])

    title   text        name
0   Person  Mark McCoy  Item 1
1   Status  None        Item 1
2   Date    2021-02-05  Item 1
3   Person              This is a new item
4   Status  None        This is a new item
5   Date    2021-04-17  This is a new item

But I think is not what you want.and there is no example to flat that kinds of array in json_normalize.

data = [ [item['name']]+[c_v['text'] for c_v in item['column_values']] for item in json_data['data']['boards'][0]['items']]
df = pd.DataFrame(data,columns=['name','Person','Status','Date'])

name    Person              Status  Date
0       Item 1  Mark McCoy  None    2021-02-05
1       This is a new item  None    2021-04-17
2       Item 5              None    2021-02-13
3       Item 2              Done    2021-05-14

so I can only flat it in python.
if you have futher question.please comment to me

Normalize Monday.com API JSON output in Python

1 Answers1