Python Pandas compare CSV keyerror

Question

I am using Python Pandas to try and match the references from CSV2 to the data in CSV1 and create a new output file.

CSV1

reference,name,house
234 8A,john,37
564 68R,bill,3
RT4 VV8,kate,88
76AA,harry ,433

CSV2

reference
234 8A
RT4 VV8

CODE

 import pandas as pd
    df1 = pd.read_csv(r'd:\temp\data1.csv')
    df2 = pd.read_csv(r'd:\temp\data2.csv')
    df3 = pd.merge(df1,df2, on= 'reference', how='inner')
    df3.to_csv('outpt.csv')

I am getting a keyerror for reference when I run it, could it be the spaces in the data that is causing the issue? The data is comma delimited.

MaxU - stand with Ukraine · Accepted Answer · 2016-04-23T10:32:05.967

most probably you have either leading or trailing white spaces in reference column after reading your CSV files.

you can check it in this way:

print(df1.columns.tolist())
print(df2.columns.tolist())

you can "fix" it by adding sep=r'\s*,\s*' parameter to your pd.read_csv() calls

Example:

In [74]: df1
Out[74]:
  reference    name  house
0    234 8A    john     37
1   564 68R    bill      3
2   RT4 VV8    kate     88
3      76AA  harry     433

In [75]: df2
Out[75]:
  reference
0     234 8A
1    RT4 VV8

In [76]: df2.columns.tolist()
Out[76]: ['reference ']

In [77]: df1.columns.tolist()
Out[77]: ['reference', 'name', 'house']

In [78]: df1.merge(df2, on='reference')

...

KeyError: 'reference'

fixing df2:

data = """\
reference 
234 8A
RT4 VV8"""
df2 = pd.read_csv(io.StringIO(data), sep=r'\s*,\s*')

now it works:

In [80]: df1.merge(df2, on='reference')
Out[80]:
  reference  name  house
0    234 8A  john     37
1   RT4 VV8  kate     88

Python Pandas compare CSV keyerror

1 Answers1