Import csv files based on condition from the folders that inside another folder

Question

I would like to copy and paste csv files based on conditions from folders inside another folder.

The main folder's name is “main”, and there are folders inside it, namely main_1, main_2, main_3,......, and main_45. There are also csv files inside each folder, namely, a1, b2, c3, d4, e5,........z30.

I just want to copy a1.csv, b2.csv, and c3.csv from the folders and paste them into another empty folder. It looks too complex to me, but I am sure it would be easy for experts. Thanks in advance

You don't need python for this. A simple shell command will do. `cp */a1.csv other_folder` And repeat for b2.csv and c3.csv. — John Gordon, Jun 13 '22 at 23:13
The main idea is curiosity about how to do that using python :) — helloworld, Jun 13 '22 at 23:14

score 1 · Accepted Answer · answered Jun 13 '22 at 23:39

os.walk iterates through directory and show all files in string.

So iterate through main folder and if the file name ends with .csv, then copy it with shutil.copyfile function

import os, shutil
for p, d, f in os.walk("main"):
    for fname in f:
        if fname.endswith(".csv"):
            shutil.copyfile(os.path.join(p, f), os.path.join("ANOTHER EMPTY FOLDER NAME", f))

docs:

https://docs.python.org/3/library/os.html

https://docs.python.org/3/library/shutil.html

`if fname in ['a1.csv', 'b2.csv', 'c3.csv']:` – John Gordon Jun 13 '22 at 23:47 — John Gordon, Jun 13 '22 at 23:47

Martin Evans · Answer 2 · 2022-06-15T11:19:16.303

If you want all CSV filenames under a certain folder and sub-folders, an easy approach would be to use Python's glob.glob() function. With this you can specify *.csv for just the CSV files and also make it recurse into sub-folders:

import glob

for csv_filename in glob.glob(r'C:\main\**\*.csv', recursive=True):
    print(csv_filename)

If you only want to copy CSV filenames which start a1 a3 or c10 for example, a regular expression could help:

import glob
import os
import shutil
import re

re_abc123 = re.compile('[a-z]\d+')

for csv_filename in glob.glob(r'C:\main\**\*.csv', recursive=True):
    basename = os.path.basename(csv_filename)
    
    if re_abc123.match(basename):
        print(csv_filename)
        shutil.copyfile(csv_filename, os.path.join('output_folder', basename))

Import csv files based on condition from the folders that inside another folder

2 Answers2