How to skip directories in os walk Python 2.7

Question

I have written an image carving script to assist with my work. The tool carves images by specified extention and compares to a hash database.

The tool is used to search across mounted drives, some which have operating systems on.

The problem I am having is that when a drive is mounted with an OS, it is searching across the 'All Users' directory, and so is including images from my local disc.

I can't figure out how to skip the 'All Users' directory and just stick to the mounted drive.

My section for os.walk is as follows:

for path, subdirs, files in os.walk(root):
    for name in files:
        if re.match(pattern, name.lower()):
                appendfile.write (os.path.join(path, name))
                appendfile.write ('\n')
                log(name)
                i=i+1

Any help is much appreciated

apologies the path i want to skip is always C:\Users\All Users — user3450524, Jul 16 '15 at 09:06

score 6 · Accepted Answer · edited Jun 20 '20 at 09:12

6

Assuming All Users is the name of the directory, you can remove the directory from your subdirs list, so that os.walk() does not iterate over it.

Example -

for path, subdirs, files in os.walk(root):
    if 'All Users' in subdirs:
        subdirs.remove('All Users')
    for name in files:
        if re.match(pattern, name.lower()):
                appendfile.write (os.path.join(path, name))
                appendfile.write ('\n')
                log(name)
                i=i+1

If you only want to not walk for All Users inside a particular parent, you can include the check for that as well in the above if condition.

From os.walk documentation -

os.walk(top, topdown=True, onerror=None, followlinks=False)

Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at directory top (including top itself), it yields a 3-tuple (dirpath, dirnames, filenames).

When topdown is True, the caller can modify the dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into the subdirectories whose names remain in dirnames; this can be used to prune the search, impose a specific order of visiting, or even to inform walk() about directories the caller creates or renames before it resumes walk() again. Modifying dirnames when topdown is False is ineffective, because in bottom-up mode the directories in dirnames are generated before dirpath itself is generated.

topdown is normally true, unless specified otherwise.

edited Jun 20 '20 at 09:12

Community

1
1

answered Jul 16 '15 at 09:06

Anand S Kumar

88,551
18
188
176

Will this skip all sub-directories within 'All Users'? – user3450524 Jul 16 '15 at 09:09
Yes it will skip sub-directories within 'All Users' as well – Anand S Kumar Jul 16 '15 at 09:10
1

@adrianus Did you even try this? It will never walk to `All Users` directory, no files or folders from it would be read – Anand S Kumar Jul 16 '15 at 09:12
1

Yes, I tried this, and all the files from the directory that is excluded still show up. – adrianus Jul 16 '15 at 09:13
that would not happen unless you have some other issue in your code. – Anand S Kumar Jul 16 '15 at 09:14
When you remove a directory from the `subdirs` name, it does not go inside it (walk inside it). – Anand S Kumar Jul 16 '15 at 09:15
I'm sorry, you're right. It will not walk though that directory and the files will get excluded, that is the correct solution. – adrianus Jul 16 '15 at 09:18
Would really like the other downvoters to comment why they think this is a bad solution. – Anand S Kumar Jul 16 '15 at 09:19

hiro protagonist · Answer 2 · 2020-06-15T05:47:15.630

2

if you have more than one directory to remove you can use a slice-assignment in oder to remove excluded directories in the subdirs

excl_dirs = {'All Users', 'some other dir'}

for path, dirnames, files in os.walk(root):
    dirnames[:] = [d for d in dirnames if d not in excl_dirs]
    ...

as the documentation states:

When topdown is True, the caller can modify the dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into the subdirectories whose names remain in dirnames; ..

edited Jun 15 '20 at 05:47

answered Apr 09 '19 at 18:32

hiro protagonist

44,693
14
86
111

why must like this [:] ? – greendino Jun 14 '20 at 20:04
@AbdullahSaid added a bit of explanation. additionally: this is a slice assignment; it modifies the list in-place (i.e. does not create a new list). – hiro protagonist Jun 15 '20 at 05:48
Thanks for this info: When topdown is True, the caller can modify the dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into the subdirectories whose names remain in dirnames; – galian Sep 15 '21 at 04:21

How to skip directories in os walk Python 2.7

2 Answers2