Os.walk wont work with directory matched with regex

Question

This is the code i used. Why cant os walk handle ?

import os, re

cwd = os.getcwd() 
directory= 'Box II'
dirpattern = re.compile(r'^.*?\\'+directory+'.*?', re.M)

for root, dirs, files in os.walk(os.path.abspath(cwd)):
    if dirpattern.search(root):
        match = dirpattern.search(root)
        match=match.group(0)

print match #OUTPUT = D:\dir1\dir2\dir3 (the directory i searched for)

for root, dirs, files in os.walk(os.path.abspath(match)):
    print files #OUTPUT = nothing

why cant i use the matched folder for a new os.walk loop?

Does this differ from your [earlier question](http://stackoverflow.com/questions/10934199/os-walk-wont-work-with-directory-acquired-through-match-group0-encodestrin)? — DSM, Jun 07 '12 at 17:46
do you come from a perl background? Do you like to self-inflict pain? — KurzedMetal, Jun 07 '12 at 17:48
Hey DSM, yes in the sense that im not manipulating the match= match.group(0) im using it as it is with os.walk. Hello KurzedMetal, yes indeed i come form a perl backround, how did you geuss cause im using regexes? — Baf, Jun 07 '12 at 17:52
@KurzedMetal why would you ask if i like seilf-inflicted pain? — Baf, Jun 07 '12 at 17:54
@DSM Do you know an answer? i see your quite adept with python — Baf, Jun 07 '12 at 17:55
Is it just me, or should the line ` print match '''OUTPUT = D:\dir1\dir2\dir3'''` raise a SyntaxError? — mgilson, Jun 07 '12 at 18:09
Everything in ''' ... ''' isnt in the actual code its just a comment from me to let you know what output i get from print. — Baf, Jun 07 '12 at 18:14
Do you have any hints or ideas mgilson? anything would be apreciatetd — Baf, Jun 07 '12 at 18:15
@Baf -- These things are usually much easier to debug if you can explain to us what you *expect* to happen and then what *is* happening. We don't have a directory structure set up like yours, so it's a little hard to pick out what the issue is. Does your script ever print a match? If not, maybe your regex isn't quite right. Do you expect to walk over more than one match in the second loop? If so, you should build a list of paths instead of just keeping one at the end of your first loop. Do you get a SyntaxError? See my previous comment ;). — mgilson, Jun 07 '12 at 18:15
@Baf -- If it's a comment, then use a python comment character ('#') — mgilson, Jun 07 '12 at 18:16
Thx for the replies mgilson. Sorry i was vague, So basically i start from the current working dir (cwd), which is where my script is. I get this with `os.getcwd()`. i initiate an os.walk with the cwd as my parameter and search for a certain dir through my compiled regex. He finds the folder im looking for alright. there are files in that folder. Now i would like to search that folder for files. through a new os.walk the parameter being the directory i searched before. this should give me all the files in that directory. but it doesnt it just nothing it doesnt even enter in the new os.walk loop — Baf, Jun 07 '12 at 18:25
Btw if this # is the comment whats that ''' ... ''' then? (i just started with python) — Baf, Jun 07 '12 at 18:30
this seems liek such a simple question, why is there no answer? :'( — Baf, Jun 07 '12 at 19:02

score 3 · Accepted Answer · answered Jun 07 '12 at 20:19

I think you want to use a generator to yield each match so that you can walk over the things that match your expression. Here is a snippet of code that I use regularly.

import os, fnmatch

#
# This wrapper for os.walk is based on Python Cookbook, 2.16.
# Added the exclude_dirs named parameter so that we can skip svn folders.
# Added the yield_files named parameter so that we can return only folders.
#

def walk_file_system (root, patterns='*', single_level=False, yield_folders=False, yield_files=True, exclude_dirs='.svn'):
    # Expand patterns from semicolon-separated string to list
    patterns = patterns.split (';')
    exclude_dirs = exclude_dirs.split (';')
    for path, subdirs, files in os.walk (root):
        for exclude_dir in exclude_dirs:
            for dir in subdirs:
                if fnmatch.fnmatch (dir, exclude_dir):
                    subdirs.remove(dir)
        fsnodes = []
        if yield_folders:
            fsnodes.extend(subdirs)
        if yield_files:
            fsnodes.extend(files)
        fsnodes.sort ( )
        for name in fsnodes:
            for pattern in patterns:
                if fnmatch.fnmatch (name, pattern):
                    yield os.path.join (path, name)
                    break
        if single_level:
            break

You can use it like this.

def walk_file_system_test():
    #patterns='*', single_level=False, yield_folders=False, yield_files=True, exclude_dirs='.svn'):
    print "Walk current directory with function defaults."
    for fsnode in walk_file_system(os.getcwd()):
        print fsnode

    #patterns='*', single_level=False, yield_folders=True, yield_files=True, exclude_dirs='.svn'):
    print "Walk current directory with yield_folders=True."
    for fsnode in walk_file_system(os.getcwd(), yield_folders=True):
        print fsnode

    #patterns='*', single_level=False, yield_folders=True, yield_files=False, exclude_dirs='.svn'):
    print "Walk current directory with yield_folders=True, yield_files=False."
    for fsnode in walk_file_system(os.getcwd(), yield_folders=True, yield_files=False):
        print fsnode

    #patterns='*', single_level=False, yield_folders=True, yield_files=False, exclude_dirs='.svn'):
    print "Walk current directory with yield_folders=True, yield_files=False, exclude_dirs='.svn;temp'."
    for fsnode in walk_file_system(os.getcwd(), yield_folders=True, yield_files=False, exclude_dirs='.svn;temp'):
        print fsnode

Hope this helps. It should be fairly simple for you to replace the fnmatch with re.

Hey ChipJust, Thx a lot for the Code u posted. Trying to understand it and testing it right now. ill get back in a sec with questions if u dont mind. — Baf, Jun 08 '12 at 08:11
Hey! didnt have time to thoroughly check the code yet, it worx like a charm tho! This is definatly a way to go, thx for that. But i still dont understand why my code doesnt work :(! — Baf, Jun 11 '12 at 13:04

Os.walk wont work with directory matched with regex

1 Answers1