Select a particular column in a CSV files with the same header and output to a new csv

Question

To explain it as simple as possible. I have multiple csv files in one directory. These csv files have temperature readings at different heights (along with other variables). I want a script that goes through all csv files and outputs each of their temperature column and date into it's own separate column a new csv called say.. "master temperature"

so let's say I have 5 files the output csv file header should look like:

date,temperature,date,temperature,date,temperature,date,temperature,date,temperature

Does this answer your question? [Concatenate rows of two dataframes in pandas](https://stackoverflow.com/questions/28135436/concatenate-rows-of-two-dataframes-in-pandas) — dzang, Mar 05 '20 at 13:57
Never use duplicates in column names if you want to later avoid headaches and maintenance nightmares... — Serge Ballesta, Mar 05 '20 at 14:25
What is the issue, exactly? Have you tried anything, done any research? Stack Overflow is not a free code writing service. See: [tour], [ask], [help/on-topic], https://meta.stackoverflow.com/questions/261592/how-much-research-effort-is-expected-of-stack-overflow-users. — AMC, Mar 05 '20 at 16:37

score 0 · Answer 1 · answered Mar 05 '20 at 14:15

I suppose you might want something like this,

import os
import pandas as pd

dfmaster = pd.DataFrame(columns=["date", "temperature"])
source = "path_to_dir_containing_csv_files"

for filename in os.listdir(source):
    fullpath = os.path.join(source, filename)
    if os.path.isfile(fullpath) and fullpath.endswith(".csv"):
        dfchild = pd.read_csv(fullpath)
        dfmaster = pd.concat([dfmaster, dfchild])

print(dfmaster.reset_index(drop=True))
dfmaster.to_csv("master.csv", index=False) #---> save the csv file

Select a particular column in a CSV files with the same header and output to a new csv

1 Answers1