remove duplicate array in multiple jsons using python

Question

I have multiple JSON records like

{
    "Stat": "DEN",
    "Change": [{
            "From": "",
            "To": "DEN",
            "changeTimestamp": "20200325000000"
        },
        {
            "From": "",
            "To": "DEN",
            "changeTimestamp": "20200325000000"
        },
    ],
    "Date": 20200401
}

since Changes array has duplicates need to eliminate them it should be like

{
    "Stat": "DEN",
    "Change": [{
            "From": "",
            "To": "DEN",
            "changeTimestamp": "20200325000000"
        }
    ],
    "Date": 20200401
}

since it is an array am unable to use list, I have a code like

doc['Changes'] = list(set(doc['Changes'])) if doc['Changes'] else []

It is working for items that are on the list but I came to know its not a list it's an array it won't work, can I get help for this, please

It is not a list like. "Changes" : [1,2,3,4]. It is array like above — srmu, Dec 14 '20 at 11:49
It is not a list like. "Changes" : [1,2,3,4]. It is array like above — srmu, Dec 14 '20 at 11:52
What do you mean by that? Perhaps you should clarify yourself with Python nomenclature. `type([1, 2, 3, 4])` gives `list`. — norok2, Dec 14 '20 at 11:57
Ok. Thanks, the actual problem for me is the above I mentioned in question is an array and if there are multiple duplicate array I need to make it as onr — srmu, Dec 14 '20 at 12:06
yes thanks so much one final question if i have json like Changes has 2 unique values(out of three 2 are duplicates) { "Status": "CLO", "Changes": [{ "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "DEC", "To": "ACT", "changeTimestamp": "20200325000000" } ], "Date": 20200401 } — srmu, Dec 14 '20 at 12:58
and once i apply this logic will the output be like this? { "Status": "CLO", "Changes": [{ "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "DEC", "To": "ACT", "changeTimestamp": "20200325000000" } ], "Date": 20200401 } — srmu, Dec 14 '20 at 12:58

score 1 · Answer 1 · answered Dec 14 '20 at 11:23

1

You can remove duplicate keys by dictionary comprehension since the dictionary does not allow duplicate keys. I have set the standard for To because you have sent it two times at the same receiver.

{each['To']: each for each in doc['Changes']}.values()

answered Dec 14 '20 at 11:23

mhhabib

2,975
1
15
29

If there are duplicates it need to correct to make one,if there are only single it should not get distrub....this will work right? – srmu Dec 14 '20 at 11:35
1

Yes it will work – mhhabib Dec 14 '20 at 11:38
Ok so it should be like doc['Changes'] ={each['To']: each for each in doc['Changes']}.values(). Am I correct? – srmu Dec 14 '20 at 11:47
Yes you're correct – mhhabib Dec 14 '20 at 11:50
yes thanks so much one final question if i have json like Changes has 2 unique values(out of three 2 are duplicates) { "Status": "CLO", "Changes": [{ "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "DEC", "To": "ACT", "changeTimestamp": "20200325000000" } ], "Date": 20200401 } – srmu Dec 14 '20 at 13:38
and once i apply this logic will the output be like this? { "Status": "CLO", "Changes": [{ "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "DEC", "To": "ACT", "changeTimestamp": "20200325000000" } ], "Date": 20200401 } – srmu Dec 14 '20 at 13:39
@srmu you shouldn't change parameters very frequently. It will make confuse what actually you want. If your problem has a solution but needs another. please set another question . – mhhabib Dec 14 '20 at 13:54
Ok I have posted new question and reverted it back – srmu Dec 14 '20 at 14:19

remove duplicate array in multiple jsons using python

1 Answers1