Compare values of a dictionary and return a count of matching values

Question

I have a dictionary comprised of product names and unique customer emails who have purchased those items that looks like this:

customer_emails = {
'Backpack':['customer1@gmail.com','customer2@gmail.com','customer3@yahoo.com','customer4@msn.com'], 
'Baseball Bat':['customer1@gmail.com','customer3@yahoo.com','customer5@gmail.com'],
'Gloves':['customer2@gmail.com','customer3@yahoo.com','customer4@msn.com']}

I am trying to iterate over the values of each key and determine how many emails match in the other keys. I converted this dictionary to a DataFrame and got the answer I wanted for a single column comparison using something like this

customers[customers['Baseball Bat'].notna() == True]['Baseball Bat'].isin(customers['Gloves']).sum()

What I'm trying to accomplish is to create a DataFrame that essentially looks like this so that I can easily use it for correlation charts.

             Backpack  Baseball Bat    Gloves
Backpack            4             2         3
Baseball Bat        2             3         1 
Gloves              3             1         3

I'm thinking the way to do it is to iterate over the customer_emails dictionary but I'm not sure how you would pick out a single key to compare its values to all others and so on, then store it.

How do you create `customers` if your dictionary lists are not equal in size? — cs95, May 14 '18 at 17:42
@cᴏʟᴅsᴘᴇᴇᴅ you have to iterate over it and turn the values into a pandas Series. `customers = pd.DataFrame(dict([ (k,pd.Series(v)) for k,v in customer_emails.items() ]))` — Trevor Theodore, May 14 '18 at 18:39
Oh... good then, I've shown you a better way to do that in my answer. — cs95, May 14 '18 at 18:40

cs95 · Accepted Answer · 2018-05-14T17:52:25.537

Start with pd.DataFrame.from_dict:

df = pd.DataFrame.from_dict(customer_emails, orient='index').T

df
              Backpack         Baseball Bat               Gloves
0  customer1@gmail.com  customer1@gmail.com  customer2@gmail.com
1  customer2@gmail.com  customer3@yahoo.com  customer3@yahoo.com
2  customer3@yahoo.com  customer5@gmail.com    customer4@msn.com
3    customer4@msn.com                 None                 None

Now, use stack + get_dummies + sum + dot:

v = df.stack().str.get_dummies().sum(level=1)
v.dot(v.T)

              Backpack  Baseball Bat  Gloves
Backpack             4             2       3
Baseball Bat         2             3       1
Gloves               3             1       3

Alternatively, switch stack with melt for some added performance.

v = (df.melt()
       .set_index('variable')['value']
       .str.get_dummies()
       .sum(level=0)
)
v.dot(v.T)

variable      Backpack  Baseball Bat  Gloves
variable                                    
Backpack             4             2       3
Baseball Bat         2             3       1
Gloves               3             1       3

score 2 · Answer 2 · answered May 14 '18 at 17:29

You can first find all the counts for each product and corresponding emails, then pass the resulting dictionary to pd.DataFrame:

import pandas as pd
emails = {'Baseball Bat': ['customer1@gmail.com', 'customer3@yahoo.com', 'customer5@gmail.com'], 'Backpack': ['customer1@gmail.com', 'customer2@gmail.com', 'customer3@yahoo.com', 'customer4@msn.com'], 'Gloves': ['customer2@gmail.com', 'customer3@yahoo.com', 'customer4@msn.com']}
results = {a:{c:sum(h in j for h in b) for c, j in emails.items()} for a, b in emails.items()}
df = pd.DataFrame(results)

Output:

               Backpack  Baseball Bat  Gloves
Backpack             4             2       3
Baseball Bat         2             3       1
Gloves               3             1       3

score 1 · Answer 3 · answered May 14 '18 at 18:07

Using the same logic create Series, then we using intersection for list

s=pd.Series(customer_emails)

pd.DataFrame(np.reshape([len(set(x).intersection(set(y)))for x in s for y in s ],(3,3)),index=s.index,columns=s.index)
Out[299]: 
              Backpack  Baseball Bat  Gloves
Backpack             4             2       3
Baseball Bat         2             3       1
Gloves               3             1       3

Compare values of a dictionary and return a count of matching values

3 Answers3