9

I have two csv files like below that I'd like to merge - more or less using the first column ID_ as the unique identifier, and append the AMT column to a new row in the final file.

CSV1

ID_ CUSTOMER_ID_    EMAIL_ADDRESS_
1090    1   example1@example.com
1106    2   example2@example.com
1145    3   example3@example.com
1206    4   example4@example.com
1247    5   example5@example.com
1254    6   example6@example.com
1260    7   example7@example.com
1361    8   example8@example.com
1376    9   example9@example.com

CSV2


ID_ AMT
1090    5
1106    5
1145    5
1206    5
1247    5
1254    65
1260    5
1361    10
1376    5

Here's what I'm looking for in a final file:

ID_ CUSTOMER_ID_    EMAIL_ADDRESS_  AMT
1090    1   example1@example.com    5
1106    2   example2@example.com    5
1145    3   example3@example.com    5
1206    4   example4@example.com    5
1247    5   example5@example.com    5
1254    6   example6@example.com    65
1260    7   example7@example.com    5
1361    8   example8@example.com    10
1376    9   example9@example.com    5

I've tried modifying a this below as much as possible, but not able to get what I'm looking for. Really stuck on this - not sure what else I can do. Really appreciate any and all help!

join -t, File1.csv File2.csv

Data shows in this example contains tabs, but my actual files are CSVs as mentioned and will contain commas as a separator.

TylerH
  • 20,799
  • 66
  • 75
  • 101
Sean Donahue
  • 93
  • 1
  • 1
  • 3
  • I ran your `join` command on my linux box and got the correct output. What system are you running? All I can think of is maybe you have a different version of `join` or the newlines are wrong. – Silas Coker Jan 26 '19 at 22:40

1 Answers1

19

This can be easily done using Pandas library. Here is my code to do this:

'''
This program reads two csv files and merges them based on a common key column.
'''
# import the pandas library
# you can install using the following command: pip install pandas

import pandas as pd

# Read the files into two dataframes.
df1 = pd.read_csv('CSV1.csv')
df2 = pd.read_csv('CSV2.csv')

# Merge the two dataframes, using _ID column as key
df3 = pd.merge(df1, df2, on = 'ID_')
df3.set_index('ID_', inplace = True)

# Write it to a new CSV file
df3.to_csv('CSV3.csv')

You can find a short tutorial on pandas here: https://pandas.pydata.org/pandas-docs/stable/getting_started/10min.html

Iman
  • 306
  • 2
  • 3