Ignore . (dot) directories in os.walk

Question

Learning a python I'm working on a generic utility script to recurse any directories below and spit filenames and other properties into a file - on Windows right now. I keep getting errors as below and I don't care about these files which appear to be found in "dot directories" so far. Manually listing directories to directories.remove() is fun whack-a-mole but I need better. I can't figure out how to just tell it to ignore all dot directories.

    file_size = os.path.getsize(filename)
  File "C:\Program Files\Python37\lib\genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'COMMIT_EDITMSG'

Here's where I am

import os
import time

# pip install --user hurry.filesize
from hurry.filesize import size

path = "./"

for root, directories, filenames in os.walk(path):
    if ".*" in directories:
        directories.remove(".*")

    for filename in filenames:
        file_path = os.path.join(root, filename)
        file_size = os.path.getsize(filename)
        better_changetime = time.strftime(
            "%Y-%m-%d %H:%M:%S", time.gmtime(os.path.getmtime(filename))
        )
        fprops = file_path + " | " + better_changetime + " | " + size(file_size)
        print(fprops)

This doesn't look like it has anything to do with dot directories. This looks like you're screwing up your path manipulation, trying to call `getsize` on the bare filename (and `".*"` doesn't act like a glob, and probably more wrong things). — user2357112, Sep 16 '19 at 18:51

Barmar · Accepted Answer · 2019-09-16T18:56:59.690

1

Filename patterns are not processed by the in operator or the .remove() method. You can use the filter() function to remove names that match a pattern.

directories = filter(lambda name: not name.startswith("."), function)

But you never use directories later, so I'm not sure what the point of this is. Removing directories from this list won't remove the files that are within them from the filenames list.

The error is because you're using filename rather than file_path when you call os.path.getsize() and os.path.getmtime().

edited Sep 16 '19 at 18:56

answered Sep 16 '19 at 18:54

Barmar

741,623
53
500
612

Python lists don't have a `filter` method. You might be mixing things up with a different language. – user2357112 Sep 16 '19 at 18:56
I always get it confused with JS. – Barmar Sep 16 '19 at 18:57
@Barmar, my scripts are throwing errors on files mostly contained in dot directories (.git, .ipynb_checkpoints for example) and I don't care about those files. So I thought I would exclude these directories to solve multiple problems. Your help resolved this error! But it highlights that I still want to exclude some directories with something like a wildcard because I now see items like "/__pycache__\pylustrator.cpython-37.pyc" and I probably don't want items from directories that startswith("__") either. – JohnZastrow Sep 16 '19 at 19:57
1

You can use `re.search()` to test against a pattern. – Barmar Sep 16 '19 at 20:00

Ignore . (dot) directories in os.walk

1 Answers1