Python Append file should refreshed with new data

Question

I am trying to write my output into a file what my code is doing is that its looking for matched file names and storing it into a file similary for unmatched files but the problem is when i use write it overwrites the file and when i use append on every run it keeps on appending the file matched file names. What i need is that it refresh te file whenever the script is run and loads it with current data only.

    import re
    import sys
    import os
    import glob
    import pandas as pd
    import logging
    
    try:
        for file in glob.glob('*.csv'):
                r = re.search(r'abc_sales_(20[0-9][0-9])-([1-9]|1[0-2]|0[0-9])-([1-9]|1[0-9]|2[0-9]|3[0-1]|0[0-9])-[0-9]{2}_[a-z0-9]{3,5}.csv', file)
                if r:
                    #matched=file
                    print(f'File matched:{file}')
                    fp=open('bad_lines.txt', 'r+')
                    sys.stdout = fp
                
                else:
                    path=f'File not matched:{file}'
                    f=open('filenotmatched.txt','a')
                    f.seek(0)
                    f.truncate()
                    f.write(path+'\n')
                    f.close()
    except Exception as e:
        pass

You're not closing the file stream by the looks of it, so start with that. Use `with open('match.txt','w') as f:` and then write to it. — Mandera, Sep 25 '20 at 08:26
@mandera did that but its overwriting my file i need all the file list not the last matched file name — Ankit Kumar Sharma, Sep 25 '20 at 08:29
@AnkitKumarSharma Can you explain in a better way which is the desired output and what you are reciving instead? — Carlo Zanocco, Sep 25 '20 at 08:34
@Carlo Zanocco so what my code is doing is its searching for files and if the file is matched it returns the output as list of matched files and list of unmatched files from else statement. So when i use `fp=open('bad_lines.txt', 'w')` it only stores the last file name and when i use `fp=open('bad_lines.txt', 'a')` on every run it stores the previous files name too which i dont want. I want new data in file on every run.For e.g for append file: a,b,c on 2nd run file:a,b,c,a,b,c desired output file: a,b,c on every run — Ankit Kumar Sharma, Sep 25 '20 at 08:37
@AnkitKumarSharma--wouldn't it be simpler to have the file opens before the for loop and write to each file depending upon the if condition? This way you would only have the result of the current run. — DarrylG, Sep 25 '20 at 08:52
@DarryIG i tried but it still overwrites the file and only stores the last file name i need the omplete list of files . The file should refresh with ne file list every time i run the script — Ankit Kumar Sharma, Sep 25 '20 at 09:19
@AnkitKumarSharma--I added an answer which illustrates what I was recommending. Is this close to what you tried? — DarrylG, Sep 25 '20 at 09:47
@DarrylG Thanks a lot it worked i really appreciate thanks a lot — Ankit Kumar Sharma, Sep 25 '20 at 09:55
@AnkitKumarSharma--glad to help. In that case, don't forget to [Accept the Answer](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work) — DarrylG, Sep 25 '20 at 10:01

DarrylG · Accepted Answer · 2020-09-25T09:52:15.403

Suggested changes to your code.

import re
import sys
import os
import glob
import pandas as pd
import logging

# We create new 'bad_lines.txt' and 
# 'filenotmatched.txt' for each run
with open('bad_lines.txt', 'w') as f_badlines, open('filenotmatched.txt','w') as f_notmatched:
    try:
        for file in glob.glob('*.csv'):
                r = re.search(r'abc_sales_(20[0-9][0-9])-([1-9]|1[0-2]|0[0-9])-([1-9]|1[0-9]|2[0-9]|3[0-1]|0[0-9])-[0-9]{2}_[a-z0-9]{3,5}.csv', file)
                if r:
                    #matched=file
                    #print(f'File matched:{file}')
                    #fp=open('bad_lines.txt', 'r+')
                    # ** Not clear why you redirected 
                    # ** standard out to a file
                    # ** rather than writing to file directly
                    #sys.stdout = fp
                    f_badlines.write(f'File matched:{file}\n')
                else:
                    path=f'File not matched:{file}'
                    #f=open('filenotmatched.txt','a')
                    #f.seek(0)
                    #f.truncate()
                    #f.write(path+'\n')
                    #f.close()
                    f_notmatched.write(path + '\n')
    except Exception as e:
        pass

Python Append file should refreshed with new data

1 Answers1