Edit dict recursively in Python

Question

I have a dict like this

{
  "library": [
    {
      "_type": "Host",
      "parameters": "JSON STRING",
      "superclassOf": [
        {
          "_type": "LinuxHost",
          "superclassOf": [
            {
              "_type": "Ubuntu",
              "superclassOf": [
                {
                  "_type": "Ubuntu1604",
                  "parameters": "JSON STRING"
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Where JSON STRING is a dict in a string form (e.g. '{"property1":"value1","property2":"value2"}'). What I am looking for is a way to navigate recoursively the supeclassOf property and convert these Json strings to real parts of json, and return the full dict once edited.

EDIT: note that supeclassOf's values are lists. So everywhere there's a superclassOf there can be multiple elements, each having (or not) properties parameters and superclassOf

EDIT2 using Prem Anand's answer I get this error:

Traceback (most recent call last):
  File "C:/Users/ceccolig/PycharmProjects/api/api.py", line 61, in <module>
    main()
  File "C:/Users/ceccolig/PycharmProjects/api/api.py", line 52, in main
    process_list_or_dict(library)
  File "C:/Users/ceccolig/PycharmProjects/api/api.py", line 45, in process_list_or_dict
    process_list_or_dict(v)
  File "C:/Users/ceccolig/PycharmProjects/api/api.py", line 45, in process_list_or_dict
    process_list_or_dict(v)
  File "C:/Users/ceccolig/PycharmProjects/api/api.py", line 45, in process_list_or_dict
    process_list_or_dict(v)
  [Previous line repeated 5 more times]
  File "C:/Users/ceccolig/PycharmProjects/api/api.py", line 43, in process_list_or_dict
    ld[k] = process_str(v)
  File "C:/Users/ceccolig/PycharmProjects/api/api.py", line 37, in process_str
    return json.loads(s)
  File "C:\Users\ceccolig\AppData\Local\Programs\Python\Python38-32\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "C:\Users\ceccolig\AppData\Local\Programs\Python\Python38-32\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\ceccolig\AppData\Local\Programs\Python\Python38-32\lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

What have you tried so far ? could you add some code you tried and didn't work ? — Hicham Zouarhi, Jul 29 '20 at 10:33
What do you mean by **"convert these Json strings to real parts of json"** ? — MisterNox, Jul 29 '20 at 10:59
@MisterNox i mean that i can access those keys. In case of a string it cannot be done — Gianluca, Jul 29 '20 at 11:03
well you can convert json data to lists/dicts so it is quite simple to access them, but give me a scenario. What should I do if I accessed them? Then I can give you an example — MisterNox, Jul 29 '20 at 11:06
@MisterNox what i need is to edit recursively the entire dict. I don't know without reading where are the keys "properties" or which entry is superclassOf another — Gianluca, Jul 29 '20 at 11:10
posted my answer right now, hope this is what you are looking for — MisterNox, Jul 29 '20 at 12:51

Prem Anand · Answer 1 · 2020-07-29T13:46:01.753

You can use json.loads to convert the json string to dict. Navigate through each element of your dict, check if it is of type dict or list or string and process each type accordingly

import json

def process_str(s):
    if s and s[0]=='{' and s[-1]=='}':
        return json.loads(s)
    return s

def process_list_or_dict(ld):
    for k,v in enumerate(ld) if isinstance(ld, list) else ld.items():
        if isinstance(v, str):
            ld[k] = process_str(v)
        elif isinstance(v, (list, dict)):
            process_list_or_dict(v)

Example

dct = {'library': [{'_type': 'Host', 'parameters': '{"property1":"value1","property2":"value2"}', 'superclassOf': [{'_type': 'LinuxHost', 'superclassOf': [{'_type': 'Ubuntu', 'superclassOf': [{'_type': 'Ubuntu1604', 'parameters': '{"property3":"value3","property4":"value4"}'}]}]}]}]}

process_list_or_dict(dct)

print(json.dumps(dct, indent=4))

Output

{
    "library": [
        {
            "_type": "Host",
            "parameters": {
                "property1": "value1",
                "property2": "value2"
            },
            "superclassOf": [
                {
                    "_type": "LinuxHost",
                    "superclassOf": [
                        {
                            "_type": "Ubuntu",
                            "superclassOf": [
                                {
                                    "_type": "Ubuntu1604",
                                    "parameters": {
                                        "property3": "value3",
                                        "property4": "value4"
                                    }
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ]
}

Thanks for the answer but I get an error. Check my question edit — Gianluca, Jul 29 '20 at 12:29
Your error means that the JSON_STRING you are using is not a valid json string. Are the individual words wrapped in double quotes as in the example string `'{"property1":"value1","property2":"value2"}'` — Prem Anand, Jul 29 '20 at 13:09
Yes I was missing a wrong string. But now it gives me this: `if s[0]=='{' and s[-1]=='}': IndexError: string index out of range` — Gianluca, Jul 29 '20 at 13:35
Just updated my answer to replace `if s[0]=='{' and s[-1]=='}':` with `if s and s[0]=='{' and s[-1]=='}':`. That should fix it. — Prem Anand, Jul 29 '20 at 13:47
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/218828/discussion-between-cristina-and-prem-anand). — Gianluca, Jul 29 '20 at 14:07
The error shows when the function parses a key whose value is '' (double sigle quote) but check s!='' doesn't work — Gianluca, Jul 29 '20 at 14:36

MisterNox · Answer 2 · 2020-07-29T14:01:32.847

Okay now I got you I hope. With reference to this question, I modified the answer so that it should work for you. Hope this is what you are looking for:

import json

data = {
  "library": [
    {
      "_type": "Host",
      "parameters": '{"property1":"value1","property2":"value2"}',
      "superclassOf": [
        {
          "_type": "LinuxHost",
          "superclassOf": [
            {
              "_type": "Ubuntu",
              "superclassOf": [
                {
                  "_type": "Ubuntu1604",
                  "parameters": '{"property1":"value1","property2":"value2"}'
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}


# if you want to modify a specific key's value where you only know the key
def replace(data, key_match, repl):
    if isinstance(data, dict):
        return {k: replace((v if k != key_match else repl), key_match, repl) for k, v in data.items()}
    elif isinstance(data, list):
        return [replace(i, key_match, repl) for i in data]
    elif isinstance(data, str):
        try:
            data = json.loads(data)
            return replace(data, key_match, repl)
        except ValueError:
            return data
    else:
        return data

print(replace(data, "property1", "Modified"))
# {'library': [{'_type': 'Host', 'parameters': {'property1': 'Modified', 'property2': 'value2'}, 'superclassOf': [{'_type': 'LinuxHost', 'superclassOf': [{'_type': 'Ubuntu', 'superclassOf': [{'_type': 'Ubuntu1604', 'parameters': {'property1': 'Modified', 'property2': 'value2'}}]}]}]}]}

EDIT

If you only want to convert the json.

def convertJSON(data):
    if isinstance(data, dict):
        return {k: convertJSON(v) for k, v in data.items()}
    elif isinstance(data, list):
        return [convertJSON(i) for i in data]
    elif isinstance(data, str):
        try:
            data = json.loads(data)
            return convertJSON(data)
        except ValueError:
            return data
    else:
        return data

print(convertJSON(data))
# {'library': [{'_type': 'Host', 'parameters': {'property1': 'value1', 'property2': 'value2'}, 'superclassOf': [{'_type': 'LinuxHost', 'superclassOf': [{'_type': 'Ubuntu', 'superclassOf': [{'_type': 'Ubuntu1604', 'parameters': {'property1': 'value1', 'property2': 'value2'}}]}]}]}]}

Hi, I don't want to replace keys' values. I just want to use json.loads where there's a string that reminds a json structure — Gianluca, Jul 29 '20 at 13:32
Then you only need to modify the replace method. I added modified method in my answer — MisterNox, Jul 29 '20 at 14:02

Edit dict recursively in Python

2 Answers2