python3: Extract same type value from Json file

Question

The following is my Json file which is decoded on base64.

response={"response": [{"objcontent": [{"title": "Pressure","rowkeys": [
                    "lat",
                    "lon",
                    "Pressure"
                ],
                "rowvalues": [
                    [
                        "WxsArK0NV0A=",
                        "uaQCWFxSM0A=",
                        "ncvggc7lcUA6MVVLnZiMQH6msaA+0yhANzLp2RsZhkBwobfXt9BXQKtxbnjV+IFARq3fVqOWiEBwyyvmt+V9QDGg7k8YUHpA4IZm9W/De0A="
                    ],
                    [
                        "WxsArK0NV0A=",
                        "HqJT4w7RUkA=",
                        "BfPox4I5ikCLVYxUxWqIQIFwlJFA+IVAJeQ6gBLyhEBB0QlkoGiCQDOkvnAZUm1AkGbWKEgza0A+FCkwH4phQHwSRSY+iVRAKcvC4pRliEA="
                    ],
                    [
                        "WxsArK0NV0A=",
                        "G5rYdw0NXkA=",
                        "C9dhhIVrg0B2hCvzOoKKQMrMWhll5o5AIujgxBB0ZkD8+EipfXx0QOXh0LLycH5ATdtxKqbtdkAw66X3l/VhQLqvZBbd13FAjKl2+8UUjUA="
                    ],
                    [
                        "WxsArK0NV0A=",
                        "PTvsm55daEA=",
                        "W+wyHC12dUCrvSLM1d6BQMfay0ZjbYpAjnk4Ecc8dkDH35pL429xQPTOwkF6Z41Aci5JATkXjUBQ6Wjlp3RQQFlpNGmsNHpAFf0DUor+dUA="
                    ]]}]}]}

I decoded the values and use these values to draw a plot.following is the code.

import base64
import struct
import numpy as np
import pylab as pl
for response_i in response['response']:
    for row in response_i['objcontent'][0]['rowvalues']:
        for item in row[:]:
            decoded=base64.b64decode(item)
            if len(decoded)<9:
                a=struct.unpack('d',decoded)

            else:
                decoded=base64.b64decode(item)
                a=struct.unpack('10d',decoded)
            last=np.array(a)
            pl.show(pl.plot(last))

but i would like to saparate the value of each list. in the 'row keys' there are 3 elements [ "lat", "lon", "Pressure"] accordingly there are 3 values in each list of rowvalues.

My question is how can I separate the different values in rowvalues and add them in each group of rowkeys. so, at the end I suppose to have 3 list which included all the decoded values.

'lat': [WxsArK0NV0A=,WxsArK0NV0A=,WxsArK0NV0A=,WxsArK0NV0A=]

'lon': [uaQCWFxSM0A=,HqJT4w7RUkA=,G5rYdw0NXkA=,PTvsm55daEA=]

'pressure': [ncvggc7lcUA6MVVLnZiMQH6msaA+0yhANzLp2RsZhkBwobfXt9BXQKtxbnjV+IFARq3fVqOWiEBwyyvmt+V9QDGg7k8YUHpA4IZm9W/De0A=, BfPox4I5ikCLVYxUxWqIQIFwlJFA+IVAJeQ6gBLyhEBB0QlkoGiCQDOkvnAZUm1AkGbWKEgza0A+FCkwH4phQHwSRSY+iVRAKcvC4pRliEA=, C9dhhIVrg0B2hCvzOoKKQMrMWhll5o5AIujgxBB0ZkD8+EipfXx0QOXh0LLycH5ATdtxKqbtdkAw66X3l/VhQLqvZBbd13FAjKl2+8UUjUA=, W+wyHC12dUCrvSLM1d6BQMfay0ZjbYpAjnk4Ecc8dkDH35pL429xQPTOwkF6Z41Aci5JATkXjUBQ6Wjlp3RQQFlpNGmsNHpAFf0DUor+dUA=] 

Answer 1

One approach would be to manually sort the data, like so:

from collections import defaultdict
from base64 import b64decode
import json

d = defaultdict(list)

js = ''
with open(json_file) as f:
    js = b64decode(f.read()).decode()
js = json.loads(js)

response = js['response']['obj_content'][0] 
for i, col_name in enumerate(response['row_keys']):
    for row_val in ['row_values']:
        d[col_name].append(row_val[i])

defaultdict automatically creates a new list when a key is called that previously didn't exist, which makes your code slightly sleeker.

Another option would be to use pandas.DataFrame and load data like so:

import pandas as pd
response = json_file['response']['obj_content'][0]
df = pd.DataFrame(response['row_values'], columns= response['row_keys'])

The neat thing about pandas is, that it's quite expansive in its features; for example, you could plot your data using the previously created DataFrame like so:

df.plot()