How to remove one of duplicate keys (one in lowercase one in uppercase) and merge their values in Python?

Question

I have a list of dictionaries called dicts. I am writing dicts into a csv file using DictWriter. dicts has duplicate keys like: Accepted Currencies (DASH) and Accepted Currencies (Dash) which I would like to merge those keys into a single key with uppercase i.e. I want to only keep Accepted Currencies (DASH) as a key and keep the value of that removed key as well; in this case keep the value of Accepted Currencies (Dash). For instance, currently, my dicts contains the following:

dicts = [{'Accepted Currencies (DASH)': 'DASH'}, {'Accepted Currencies (Dash)': 'Dash'}]

But I want to have something like the following:

dicts = [{'Accepted Currencies (DASH)' : 'DASH'}, {'Accepted Currencies (DASH)': 'Dash'}]

Here is my code:

dicts = []
    for j in range(1,39):
        for i in range(1,10):


###Some code here to calculate super_dict

            super_dict.update(social_urls)

            super_dict.update(metadata)
            super_dict.update(project_name)
            super_dict.update(bc_dict)
            super_dict.update(ind_dict)
            super_dict.update(pos_dict)
            super_dict.update(likes_dict)
            super_dict.update(memprof_dict)
            super_dict.update(video_link)
            super_dict.update(details_dict)
            dicts.append(super_dict)

Please explain what you mean by "also keep both keys' values." A Python `dict` can have only one value for a given key. It is possible (and common) to make that value a list which can hold multiple values inside the list. Is that what you mean? To be consistent, would you want to change the values of non-duplicated keys to hold a list containing only one value? Please clarify. Also, you show no `dict` in your code--you refer to `dicts` but never show its definition. Please show a complete, runnable code snippet. — Rory Daulton, Jun 24 '19 at 21:23
@Rory Daulton: No, I mean I want to merge duplicate keys into one key with uppercase and then keep their values as well. I don't want to store the valuse into a list. I'd like to have them in separate cells. Something like in the image above. Thanks. — Zeinab Akhavan, Jun 24 '19 at 21:30
It is still unclear. What do you mean by "store the value of that removed key." Store it where? What is the meaning of the spreadsheet you show--it certainly does not look like a csv file, since the rows are inconsistent. What does "separate cells" mean? And so on. — Rory Daulton, Jun 24 '19 at 21:43
Your test case is unclear. Please eliminate unrelated stuff. You should only need two key-value pairs in the before, and one in the after. I'm not sure how html parsing or writing a csv matters to the merged dict, so please clarify. — Kenny Ostrom, Jun 24 '19 at 21:51
@Rory Daulton: Sorry for the confusion. I added two separate images of what I have right now and what I would like to get. Thanks. — Zeinab Akhavan, Jun 24 '19 at 21:52
@ Kenny Ostrom: super_dict gets updated in each iteration of the for loops and then I append it to a list. Then I try to write that list of dictionaries into a csv file but before doing that I'd like to remove duplicate keys and merge their values. — Zeinab Akhavan, Jun 24 '19 at 21:59
Can you provide the input and output. We don't need PNG's, I can't see them and I can't copy/paste them into my python environment. And we can skip the csv because it doesn't matter. You will write a dict to the csv but it has nothing to do with the question here. — Kenny Ostrom, Jun 24 '19 at 22:28
The code is not relevant. It would be better if you just removed it. — wjandrea, Jun 24 '19 at 23:04
So is this just an example, or do you only want to replace the key `'Accepted Currencies (Dash)'`? If it's just an example, please clarify how you determine which keys in which dicts to change and which to keep. And please do an actual attempt at solving this yourself and add it in, cause that can help indicate your intention and what you're having difficulty with. — wjandrea, Jun 24 '19 at 23:15
Ah, you want to normalize the key names. Have you considered defining a class? — Kenny Ostrom, Jun 25 '19 at 13:22
@Kenny Ostrom: Yes, exactly. No, I don't know how to do that. — Zeinab Akhavan, Jun 25 '19 at 19:55
You could just force the keys to all uppercase, or all lowercase. It's more complicated if you want to enforce some weird rules like "camel-case except all uppercase inside parentheses" but if you want that, you have to write a function to do it. Then you rename all the keys using your normalization function. — Kenny Ostrom, Jun 25 '19 at 20:01
@Kenny Ostrom: Thank you. Can you please give me a hint for this particular example above that how I can do it? — Zeinab Akhavan, Jun 25 '19 at 20:08

Kenny Ostrom · Answer 1 · 2019-06-25T20:20:12.803

Essentially, you have inconsistent naming in the dictionary keys, so you want to make them follow the same guidelines. Often called normalization: https://en.wikipedia.org/wiki/Normalization

The easiest rule to implement is just "make them all lowercase", although you could write your own rules, for example "initial capital on all words, except all caps in parentheses". I just provide the all lowercase implementation here.

# define a function that makes all key names follow the same guidelines (normalization)
def normalize_keyname(key):
    return key.lower()

before = [
    {'Accepted Currencies (DASH)': 'DASH'}, 
    {'Accepted Currencies (Dash)': 'Dash'}
]

after = [
    {normalize_keyname(key): value for key, value in entry.items()} 
    for entry in before
]

for entry in after:
    print(entry)

{'accepted currencies (dash)': 'DASH'}
{'accepted currencies (dash)': 'Dash'}

I focused on the technique of making them have consistent key names. Tou can improve readability by changing the implementation details in the normalize_keyname function, but I'd like to leave that part to you. Although making it pretty isn't a concern in the data -- you can always handle that in a formatter when you display it.

score 0 · Answer 2 · answered Jun 25 '19 at 21:40

def normalize_keyname(key):
    if ' (' in key:
        ixx = key.find(' (')
        tmp = key[ixx:].upper()
        cap_key = key[:ixx]+tmp


    else:
        cap_key = " ".join(w.capitalize() for w in key.split())
    return cap_key

before = [
    {'Accepted Currencies (DASH)': 'DASH'}, 
    {'Accepted Currencies (Dash)': 'Dash'}
]

after = [
    {normalize_keyname(key): value for key, value in entry.items()} 
    for entry in before
]

for entry in after:
    print(entry)

How to remove one of duplicate keys (one in lowercase one in uppercase) and merge their values in Python?

2 Answers2