0

I am trying to do some work within some files in a directory. The basic structure of what I'm trying to work with is folder -> sub-folders -> files I need to access. data holds hundreds of subfolders, I am trying to access each one, find the file within them that ends in 'params', and for now just read the contents. My code is below:

import os
for sub_folder in os.scandir('data'):
    os.chdir(sub_folder)
    for file in os.scandir(sub_folder):
        print(file.name)
        if(file.name.endswith('params')):
            with open(file.name, 'r') as f:
                data = f.read()

I'm getting a FileNotFoundError, where it's telling me that the path 'data\\\run.0' doesn't exist. I have confirmed that 'run.0' is the first sub folder within data, so where I'm confused is how the path doesn't actually exist.

I know the error is happening when I attempt to change directories, so I'm suspecting the way that I am traversing the data folder is not a correct way of doing so. I understand that os.scandir gives a DirEntry object, which is what the variable sub_folder will be but is this not a valid input for the change directory function?

martineau
  • 119,623
  • 25
  • 170
  • 301
WeekendJedi
  • 67
  • 10
  • 1
    You don't change dir back to the base directory after each loop, so is it looking inside the first folder it found for the second? – ch4rl1e97 Oct 08 '21 at 21:35
  • In the second scandir loop, you are not `chdir`'ing into the subfolder, so the current directory is not the one where `file.name` lives. – John Gordon Oct 08 '21 at 21:46

1 Answers1

0

You can use os.walk, but I prefer use glob: See How to use Glob() function to find files recursively in Python?

martineau
  • 119,623
  • 25
  • 170
  • 301
Devyl
  • 565
  • 3
  • 8
  • In order to be immediately helpful to readers (and avoid [link-rot](http://en.wikipedia.org/wiki/Link-rot)), answers that provide at least a summary of the solution directly are preferred, with links used to offer additional information. – martineau Oct 08 '21 at 22:52