Python - replace part of the string inside of all files within given directory with previously splitted string

Question

I have a lot of files inside of directory containing some paths. Need to go through all files, and replace first part of the "path" with other one which is I previously got. Have in mind that 'sw' will always be a border between those 2 parts after split.

So inside of the files paths are:

E:\p\folder\folder2\sw\first_step\step1\step2\final
E:\p\folder\folder2\sw\second_step\step1\step2\final
E:\p\folder\folder2\sw\second_step\step56\step333\final
E:\p\folder\folder2\sw\second_step\step1\step2\final
etc

My code:

def findReplace(directory, find, replace, filePattern):
    for path, dirs, files in os.walk(os.path.abspath(directory)):
        for filename in fnmatch.filter(files, filePattern):         
            filepath = os.path.join(path, filename)
            with open(filepath) as f:
                s = f.read()
            s = s.replace(find, replace)
            with open(filepath, "w") as f:
                f.write(s)

base_dir = 'F:/Users/User1/Folder/'
path_to_files = '/folder1/folder2/files/'

directory = os.listdir(os.path.join(base_dir, path_to_includes_appl))

for file in directory:
    if file == None: continue
    with open (file, 'r') as f:
        content = f.read().split('\n')
        for line in content:
            if not line: continue
            # Idea here is to sepeare part we need to replace with part that has to stay
            old_path, path_to_file = line.split('sw\\')
            

findReplace(str(directory), old_path, base_dir, ".c")

Final output should be:

F:\Users\User1\Folder\sw\first_step\step1\step2\final
F:\Users\User1\Folder\second_step\step1\step2\final
F:\Users\User1\Folder\second_step\step56\step333\final
F:\Users\User1\Folder\second_step\step1\step2\final

NOTE: Please ignore diff between "/" and "\"

Anyone has idea what am I doing wrong, or has some better idea of how to achieve this? Thanks!

EDIT:

This is my new code:

base_dir = 'F:/Users/User1/Folder/'
path_to_includes_appl =  '/folder1/folder2/files/'
# directory = os.listdir(os.path.join(base_dir, path_to_includes_appl))
directory_path = (os.path.join(base_dir, path_to_includes_appl))
directory = os.listdir(directory_path)


for file in directory:
        with open (directory_path+file, 'r') as f:
            content = f.readlines()
            for line in content:
                old_path, path_to_file = line.split('sw\\')
                findReplace(directory_path, old_path, base_dir, ".c")

When I run this, nothing happens, no error but also no updated files

We can't run it so you have to describe what you get. If you get error message then you should show FULL error in question (not in comment). We can't read in your mind. — furas, Oct 14 '21 at 16:28
maybe first use `print()` to see what you have in variables. It is called `"print debuging"` and it can show you where is problem. — furas, Oct 14 '21 at 16:30
Better read all file, close it, change path in text and open file for writing and save all text in file. — furas, Oct 14 '21 at 16:32
you have wrong indentation - you should run `findReplace()` inside `for`-loop (directly after you split code) but you run it after `for`-loop - so you run it only for last value. — furas, Oct 14 '21 at 16:33
in `directory` you have list with all files in folder but later you convert it to string - `str(directory)` and you try to use it inside `findReplace` and this is other big mistake - it looks like you expect single path in this variable. It seems like you use missleading name `directory` for your variable. If you would use `print()` to see what you have in variables then you would see this problem. — furas, Oct 14 '21 at 16:36
as I know `listdir()` can't get list with `None` so checking `if file == None:` is useless. Besides there is preferred `if file is None:` - eventually `if not file:` — furas, Oct 14 '21 at 16:38
Thank you for pointing to these issues, I will update now my question with current status — John, Oct 15 '21 at 10:13

score 1 · Answer 1 · answered Oct 15 '21 at 13:19

1

So I have managed to solve this problem, here is the working code:

directory_path = (os.path.join(base_dir, path_to_includes_appl))
directory = os.listdir(directory_path)

for file in directory:
    with open (directory_path+file, 'r') as f:
        content = f.readlines()
    f.close()
    with open (directory_path+file, 'w') as fw:
        for line in content:
            path = pathlib.Path(line)
            index = path.parts.index('sw')
            new_path = pathlib.Path(base_dir).joinpath(*path.parts[index:])
            fw.write(str(new_path))
        fw.close()

answered Oct 15 '21 at 13:19

John

230
2
12

if you use `with` to open file then you don't need `close()` because `with` automatically closes file. – furas Oct 15 '21 at 16:16
instead of `directory_path+file` you could use `os.path.join(directory_path, file)`. I see you use `Path` - so you could create `BASE = pathlib.Path(base_dir)` at start and later you could use `/` to create full path - `directory_path = BASE / path_to_includes_appl`. And `new_path = BASE.joinpath(...)`. And you could even use `for file in directory_path.iterdir()` and `file.absolute()` instead of `directory_path+file`, or ever `file.read_text()` and `file.write_text()` but this reads and write all at once. – furas Oct 15 '21 at 16:25
thanks man, this comment helps me in other problem I have, which is when I define like this on Azure pipeline: base_dir = os.getenv('AGENT_BUILDDIRECTORY'), and then use it after in path = os.path.join(base_dir + something), I never get full path but just "C:" – John Oct 19 '21 at 08:38

Python - replace part of the string inside of all files within given directory with previously splitted string

1 Answers1