Multiple relational tables to nested JSON format using Python

Question

I'm trying to create nested JSON object by combining more than one relational tables using python/pandas. I'm a beginner in Python/pandas, so looking for bit of a help here...

In the following example, instead of tables, I'm using CSV files just to keep it simple

Table1.csv

Emp_id, Gender, Age
1, M, 32
2, M, 35
3, F, 31

Table2.csv

Emp_id, Month, Incentive
1, Aug, 3000
1, Sep, 3500
1, Oct, 2000
2, Aug, 1500
3, Aug, 5000
3, Sep, 2400

I want to create a JSON object like below

*Required output:

{
    "data": [{
        "employee": 1,
        "gender": M,
        "age": 32,
        "incentive": [{
            "aug": 3000,
            "sep": 3500,
            "oct": 2000
        }],
        "employee": 2,
        "gender": M,
        "age": 35,
        "incentive": [{
            "aug": 1500
        }],
        "employee": 3,
        "gender": F,
        "age": 31,
        "incentive": [{
            "aug": 5000,
            "sep": 2400
        }]
    }]
}

Answer 1

Use merge with left join first, then groupby with lambda function for dictionaries and convert to_dict, last add top key value and convert to json:

d = (df1.merge(df2, on='Emp_id', how='left')
         .groupby(['Emp_id','Gender','Age'])['Month','Incentive']
         .apply(lambda x: [dict(x.values)])
         .reset_index(name='Incentive')
         .to_dict(orient='records')

)
#print (d)

import json
json = json.dumps({'data':d})

---

print (json)

{
    "data": [{
        "Emp_id": 1,
        "Gender": "M",
        "Age": 32,
        "Incentive": [{
            "Aug": 3000,
            "Sep": 3500,
            "Oct": 2000
        }]
    }, {
        "Emp_id": 2,
        "Gender": "M",
        "Age": 35,
        "Incentive": [{
            "Aug": 1500
        }]
    }, {
        "Emp_id": 3,
        "Gender": "F",
        "Age": 31,
        "Incentive": [{
            "Aug": 5000,
            "Sep": 2400
        }]
    }]
}