0

i have tried to solve my problem studing this and this and several other pages but i'm not able to understand. I hope i will ask something new.

I try to explain the problem because i don't known the right way to solve it.

I make two requests with requests.get to obtain two json answer from the same api of two version of the same sdk. I want to understand if those json are equal but i want to "whitelist" some values thats i known it will be different (for example creation timestamp).

To do this i want to replace some values of the same keys with the same values in order to ignore difference inside some keys.

For example /server/v1/api.php response is :

{
    "result" : "OK",
    "data" : {
        "text" : "someText",
        "array" : [
            "foo", "bar"
        ],
        "dateTime" : "24/10/2015 22:14"
    }
}

and /server/v2/api.php response is :

{
    "result" : "OK",
    "data" : {
        "text" : "someText",
        "array" : [
            "foo", "bar"
        ],
        "dateTime" : "24/10/2015 22:15"
    }
}

Now I want to "whitelist" dateTime thats because it's normal that it values will change.

How to replace values of a key at every deep? I need to obtain

{
    "result" : "OK",
    "data" : {
        "text" : "someText",
        "array" : [
            "foo", "bar"
        ],
        "dateTime" : "whitelisted"
    }
}

so the two json will be equal.

in the real case i DON'T known json structure and I NEED to USE a hardcoded whitelist to filter the test result so i need to iterate over the json in order to apply the whitelist.

this is a real example

d1 = {
   "ssoCode": "OK",
   "errorDescription": "",
   "content": {
        "xcontentId": "508e5d95-2267-4408-b801-3742819a6d98",
        "creationDate": "2015-10-26T16: 16: 07.504Z",
        "sourcefilesOldVersions": [],
        "id": "b37412ee-b7de-4642-9d38-ae336e030f77",
        "userId": "andrea.bisello",
        "owner": "twin",
        "contentType": "PAGELET",
        "solution": "VIEW",
        "availableInSolutions": [
            "VIEW"
        ],
        "sourceFiles": [],
        "packagedId": [],
        "channels": [
            {
                "status": "PUBLISHED",
                "channelType": "WEB",
                "startTime": "2015-10-26T16: 16: 07.707Z",
                "finishTime": "2015-10-26T16: 16: 07.723Z"
            }
        ],
        "xpublishedId": "0ad653cc-664a-4d36-8b9e-b7788a43a3b4"
    },
    "resultCode": "OK",
    "actionsInError": []
}

update :

as suggested by Letzerwille i should use dict.update() to apply whitelist. How to apply to dict contained in my first dict without removing other keys? for example this is my whitelist dict

whitelist = {
    "content": {
        "xcontentId": "whitelisted",
        "creationDate": "whitelisted",
        "id": "whitelisted",
        "channels": [
            {
                "startTime": "whitelisted",
                "finishTime": "whitelisted"
            }
        ],
        "xpublishedId": "whitelisted"
    }
}

it keeps all the keys that i want "to whitelist".

but applying this dict i will destroy every other keys of "content" dict, because the entire "content" keys will be overwritten by the new one so i lost, for example, ["content"]["owner"] that should not be removed (this is the result)

{
    "ssoCode": "OK",
    "actionsInError": [],
    "errorDescription": "",
    "resultCode": "OK",
    "content": {
        "creationDate": "whitelisted",
        "id": "whitelisted",
        "xcontentId": "whitelisted",
        "xpublishedId": "whitelisted",
        "channels": [
            {
                "startTime": "whitelisted",
                "finishTime": "whitelisted"
            }
        ]
    }
}

how i can solve this problem?

so i tried to make a function with receive a dict and a whitelist (for example whitelist = ["id"] ) but something during the iteration of the list doesn't work and it return a corrupted object (thats because entering in the list switch case, the function exit)

def applyWhitelist(something, whitelist):
    print(something)
    if isinstance(something, dict):
        for k, v in something.items():
            print(k, v)
            if isinstance(v, str):
                if k in whitelist:
                    something[k] = "whitelisted"
            if isinstance(v, int):
                if k in whitelist:
                    something[k] = "whitelisted"
            if isinstance(v, dict):
                return applyWhitelist(something[k], whitelist)
            if isinstance(v, list):
                if len(v) > 0:
                    return applyWhitelist(something[k], whitelist)
    if isinstance(something, list):
        for el in something:
            if isinstance(el, dict):
                return applyWhitelist(el, whitelist)
            if isinstance(el, list):
                if len(el) > 0:
                    return applyWhitelist(el, whitelist)
            if isinstance(el, str):
                if el in whitelist:
                    el = "whitelisted"
            if isinstance(el, int):
                if el in whitelist:
                    el = "whitelisted"
    if isinstance(something, str):
        if something in whitelist:
            something = "whitelisted"
    if isinstance(something, int):
        if something in whitelist:
            something = "whitelisted"

Is my way the best way to obtain that result? any suggestions?

Community
  • 1
  • 1
Andrea Bisello
  • 1,077
  • 10
  • 23

1 Answers1

2
    to whitelist use update. say you what to whitelist d.

     d = {
           "ssoCode": "OK",
           "errorDescription": "",
           "content": {
                "xcontentId": "508e5d95-2267-4408-b801-3742819a6d98",
                "creationDate": "2015-10-26T16: 16: 07.504Z",
                "sourcefilesOldVersions": [],
                "id": "b37412ee-b7de-4642-9d38-ae336e030f77",
                "userId": "andrea.bisello",
                "owner": "twin",
                "contentType": "PAGELET",
                "solution": "VIEW",
                "availableInSolutions": [
                    "VIEW"
                ],
                "sourceFiles": [],
                "packagedId": [],
                "channels": [
                    {
                        "status": "PUBLISHED",
                        "channelType": "WEB",
                        "startTime": "2015-10-26T16: 15: 07.707Z",
                        "finishTime": "2015-10-26T16: 16: 07.723Z"
                    }
                ],
                "xpublishedId": "0ad653cc-664a-4d36-8b9e-b7788a43a3b4"
            },
            "resultCode": "OK",
            "actionsInError": []
        }

    change the value that need to be whitlisted. Here I have changed  "startTime" to "=========================="


        d_update = {
           "ssoCode": "OK",
           "errorDescription": "",
           "content": {
                "xcontentId": "508e5d95-2267-4408-b801-3742819a6d98",
                "creationDate": "2015-10-26T16: 16: 07.504Z",
                "sourcefilesOldVersions": [],
                "id": "b37412ee-b7de-4642-9d38-ae336e030f77",
                "userId": "andrea.bisello",
                "owner": "twin",
                "contentType": "PAGELET",
                "solution": "VIEW",
                "availableInSolutions": [
                    "VIEW"
                ],
                "sourceFiles": [],
                "packagedId": [],
                "channels": [
                    {
                        "status": "PUBLISHED",
                        "channelType": "WEB",
                        "startTime": "==========================",
                        "finishTime": "2015-10-26T16: 16: 07.723Z"
                    }
                ],
                "xpublishedId": "0ad653cc-664a-4d36-8b9e-b7788a43a3b4"
            },
            "resultCode": "OK",
            "actionsInError": []
        }

    Now run update command on dictionary you want to make that change.

    d.update(d_update)

    pp(d)


    {'actionsInError': [],
     'content': {'availableInSolutions': ['VIEW'],
                 'channels': [{'channelType': 'WEB',
                               'finishTime': '2015-10-26T16: 16: 07.723Z',
                               'startTime': '==========================',
                               'status': 'PUBLISHED'}],
                 'contentType': 'PAGELET',
                 'creationDate': '2015-10-26T16: 16: 07.504Z',
                 'id': 'b37412ee-b7de-4642-9d38-ae336e030f77',
                 'owner': 'twin',
                 'packagedId': [],
                 'solution': 'VIEW',
                 'sourceFiles': [],
                 'sourcefilesOldVersions': [],
                 'userId': 'andrea.bisello',
                 'xcontentId': '508e5d95-2267-4408-b801-3742819a6d98',
                 'xpublishedId': '0ad653cc-664a-4d36-8b9e-b7788a43a3b4'},
     'errorDescription': '',
     'resultCode': 'OK',
     'ssoCode': 'OK'}


to find the value which is different use:

def dict_comp(d1, d2):
    # compare dicts with identical structure
    # return True if identical values,
    # return False if a value is different
    if d1 == d2:
        return True
    else:
        for (k1, v1), (k2, v2) in zip(sorted(d1.items()), sorted(d2.items())):
            if k1 == k2:
                if isinstance(v1, dict):
                    return dict_comp(v1, v2)
                else:
                    if v1 == v2:
                        pass
                    else:
                        print("different values found at key --> "
                              "{} <-- {} {} ".format(k1, v1, v2))
                        return False
LetzerWille
  • 5,355
  • 4
  • 23
  • 26
  • thanks. with you suggestion i created a function that cover my use case, so i edited the first post. it is harder because **response structure is not fixed so i need to iterate over the element, but iteration fail**. – Andrea Bisello Oct 27 '15 at 11:17
  • @AndreaBisello you problem is in two parts. 1.to whitlist 2. to see if the values are different. To find a value that is different you can you the func that i wrote. to whitelist nothing should be invented. you can use update command. For example take a copy of you dict name is d_update and replace value in the d_update of startTiem to "===================' Now all you need to whitelist any dict of the same structure. You need just to do d_that_needed_to_be_white_listed.update(d_update) – LetzerWille Oct 27 '15 at 14:16
  • @LetzeWille update looks great, i study documentation but with "update" i will "overwrite" the keys with the new value so my whitelist dict will "destroy" the other keys of "content" . i updated question – Andrea Bisello Oct 27 '15 at 16:26
  • @LetzeWille sorry, i will explain myself better : whitelist can be loaded ! i cannot manually whitelist every json. my test suite contains the GET/POST request and the whitelist, so it will ask for result the two server and use the whitelist to remove keys that can be different. anyway thanks for you big help. – Andrea Bisello Oct 27 '15 at 16:50