How to remove the same pair with different order in values inside dictionary in Python?

Question

I have a list of dictionary that look like that

 {'ip_src': '2.2.2.2',
  'ip_dst': '1.1.1.1',
  'src_id': 43,
  'src_name': 'test1',
  'dst_id': 48,
  'dst_name': 'test2'},

 {'ip_src': '1.1.1.1',
  'ip_dst': '2.2.2.2',
  'src': 48,
  'src_name': 'test2',
  'dst': 43,
  'dst_name': 'test1'},


 {'ip_src': '4.4.4.4',
  'ip_dst': '3.3.3.3',
  'src_id': 41,
  'src_name': 'test1',
  'dst_id': 47,
  'dst_name': 'test2'},

 {'ip_src': '3.3.3.3',
  'ip_dst': '4.4.4.4',
  'src': 47,
  'src_name': 'test2',
  'dst': 41,
  'dst_name': 'test1'},

i want to remove the duplicate data connection because the src and dst maybe diffrent but its on the same cable.

so i want that my list will look like this:

 {'ip_src': '2.2.2.2',
  'ip_dst': '1.1.1.1',
  'src_id': 43,
  'src_name': 'test1',
  'dst_id': 48,
  'dst_name': 'test2'},


 {'ip_src': '4.4.4.4',
  'ip_dst': '3.3.3.3',
  'src_id': 41,
  'src_name': 'test1',
  'dst_id': 47,
  'dst_name': 'test2'},

I try to compare every item in the list and make a reverse but it didn't work because after I look into it, I realised that the order of the items in the dictionary must be 100% reverse and not only the value.

Does someone have an idea how to solve my problem?

i was looking also at those solution: https://stackoverflow.com/questions/56060973/removing-inverse-duplicates-in-dictionary-python https://stackoverflow.com/questions/8749158/removing-duplicates-from-dictionary — Ndor, Sep 12 '21 at 13:47
What code have you already tried? Where are you getting stuck? — BLimitless, Sep 12 '21 at 13:51
Explain the logic of finding a duplicate. And explain which entry should be in the outout. — balderman, Sep 12 '21 at 13:54
@buran i decide which dict to remain if the src_id and dst_id are the same but opposite — Ndor, Sep 12 '21 at 13:54
your keys are not consistent sometimes it is `ipo_src` sometimes it is `ip_src` — Epsi95, Sep 12 '21 at 13:55
@BLimitless i tried to used this solution https://stackoverflow.com/questions/8749158/removing-duplicates-from-dictionary i make a for loop inside a for loop and check the src_id and the dst_id if are the same but opposite — Ndor, Sep 12 '21 at 13:56
@Ndor, I understand that, but if `src_id` and `dst_id` are oposite, same apply for `ip_src` and `ip_dst`. My questions is how you decide which of the two duplicate dicts to remain. — buran, Sep 12 '21 at 14:01
@buran i decide wich one of the two to remove if the ids/ips are the same and opposite because if A connect to B and B connect to A both are on the same cable its dosnt metter which one u remove A->B or B->A in the end its give me the same view(in this case double same view) — Ndor, Sep 12 '21 at 14:09
@Ndir, from your example - you keep dicts at index 0 and 2, why not e.g. 1 and 3. I.e. wyy 0 not 1 and why 2 not 3. I understand the logic why 0 and 1 are duplicate and you want one of them removed. — buran, Sep 12 '21 at 14:10

score 1 · Accepted Answer · answered Sep 12 '21 at 13:58

There are many ways to do it, but simplest one is to convert it to dict with (src_ip, dest_ip) as keys, by dict property duplicates will be removed.

l = [{'ip_src': '2.2.2.2',
  'ip_dst': '1.1.1.1',
  'src_id': 43,
  'src_name': 'test1',
  'dst_id': 48,
  'dst_name': 'test2'},

 {'ip_src': '1.1.1.1',
  'ip_dst': '2.2.2.2',
  'src': 48,
  'src_name': 'test2',
  'dst': 43,
  'dst_name': 'test1'},


 {'ip_src': '4.4.4.4',
  'ip_dst': '3.3.3.3',
  'src_id': 41,
  'src_name': 'test1',
  'dst_id': 47,
  'dst_name': 'test2'},

 {'ip_src': '3.3.3.3',
  'ip_dst': '4.4.4.4',
  'src': 47,
  'src_name': 'test2',
  'dst': 41,
  'dst_name': 'test1'}]

temp_l = {tuple(sorted([i['ip_src'], i['ip_dst']])): i for i in l}
final = list(temp_l.values())

print(final)

# [{'ip_src': '1.1.1.1',
#   'ip_dst': '2.2.2.2',
#   'src': 48,
#   'src_name': 'test2',
#   'dst': 43,
#   'dst_name': 'test1'},
#  {'ip_src': '3.3.3.3',
#   'ip_dst': '4.4.4.4',
#   'src': 47,
#   'src_name': 'test2',
#   'dst': 41,
#   'dst_name': 'test1'}]

Hey man thank you very much its work lick a charm. can u explain play how this code work? and why? — Ndor, Sep 12 '21 at 14:04
I already mentioned in the post, but to tell it is that we are creating a python dictionary with keys ('ip_src', 'ip_dst') tuple, if multiple elements has the same key, the old value will be replaced (property of dictionary). Try doing like `d = {(1,2): 'value', (1,2): 'another value'}` then `print(d)` — Epsi95, Sep 12 '21 at 14:08

How to remove the same pair with different order in values inside dictionary in Python?

1 Answers1