Find all files with os.walk for a specific directory only

Question

A directory tree looks like this:

DirA--
     |
     -- Map
     |
     -- Fig--
            |
            --file.png
     |
     -- Data--
              |
              -- file.xls
              |
              -- file.csv

There are multiple directories, containing multiple files. I would like to get full path to those files that are found in Data directory only.

This is what I have so far:

dirlist = []
thisdir = os.getcwd()

for root, dirs, files in os.walk(thisdir):
    for d in dirs:
        if d.startswith("Data"):
            dirlist.append(os.path.join(root, d))

try `thisdir = os.getcwd() + "/Data"`. And after that loop over file as usual — jacob galam, Sep 25 '20 at 11:22
thanks, os.getcwd() is pointing to the root, within the root I have several directories Dir1, Dir2, etc and "Data" is a subdirectory. This wont work I am afraid. — olyashevska, Sep 25 '20 at 11:36
I don't think I understand. getcwd() returns the current working directory of a process, which it the root folder? "Data" folder is in the root? — jacob galam, Sep 25 '20 at 11:42
sorry @jacobgalam, maybe I am confused myself. Where exactly do you suggest adding it? — olyashevska, Sep 25 '20 at 11:58
Thanks for the clarification, os.getcwd() is pointing to the directory which contain DirA, DirB, DirC, DirD, each of them may or may not contain Data. The full path would be os.getcwd() + "/DirA" + "/Data". Does it make sense? — olyashevska, Sep 25 '20 at 12:05

Faizan · Accepted Answer · 2020-09-25T12:30:58.733

3

To get only Data directory files, you will need to combine root and files.

for root, dirs, files in os.walk(thisdir):
        if "Data" in root: # try using in instead of startswith
           for f in files:
            dirlist.append(os.path.join(root, f))

Trying to do it using 'dirs' In case of 'dirs', you don't have access to the files. For example, when root is DirA, you will have --Data-- in your dirs list but you will have no access to the files of --Data-- folder.

edited Sep 25 '20 at 12:30

answered Sep 25 '20 at 11:50

Faizan

268
2
9

this looks very neat, however it returns an empty list. It should be if dirs.startswith("Data"), no? – olyashevska Sep 25 '20 at 11:57
No, you can not do that. – Faizan Sep 25 '20 at 12:23
1

aha, changing root.startswith() to root.endswith() worked! Because --Data-- will be at the end of the path not start – olyashevska Sep 25 '20 at 12:27

jacob galam · Answer 2 · 2020-09-25T12:32:40.193

0

import os
from os import listdir
from os.path import isfile, join

rootdir = os.getcwd()

folder_name = "Data"

def get_files(path):
    onlyfiles = [f for f in listdir(path) if isfile(join(path, f))]
    return onlyfiles

def get_search_files(start_path, folder_name):
    for subdir, dirs, files in start_path:
        for x in dirs:
            if x == folder_name:
                data_folder_path = os.path.join(subdir, x)
                dirlist = get_files(data_folder_path)
                return dirlist

dirlist = get_search_files(os.walk(rootdir), folder_name)

edited Sep 25 '20 at 12:32

answered Sep 25 '20 at 12:02

jacob galam

781
9
21

Thanks, but it gives an empty set. – olyashevska Sep 25 '20 at 12:19
1

thanks, I accepted other answer because it is very concise. – olyashevska Sep 25 '20 at 12:39

Find all files with os.walk for a specific directory only

2 Answers2