Opening all files in a directory for CSV and reading each line Python

Question

Hi I am looking for some help with regards to a problem I am having. I want to search the directory that my data is in (shown below) for only certain file types. Below is my code but it is not quite functioning.

The current output is one of two results. Either it prints just the first line of the file, just prints blank result.

OK so here is what I want to do. I want to search the directory listed for only csv files. Then what I want to do is to get the loop to read each file line by line and print each line in the file and then repeat this for the rest of the csv files.

Can anyone please show me how to edit the code below to search only for CSV files and also how to print each line that is in the file and then repeat for the next CSV file untill all the CSV files are found and opened. Is this possible?

import os

rootdir= 'C:\Documents and Settings\Guest\My Documents\Code'

def doWhatYouWant(line):
    print line

for subdir, dirs, files in os.walk(rootdir):
   for file in files:
        f=open(file,'r')
        lines = f.readlines()
        f.close()
        f=open(file,'wU')
        for lines in lines:
            newline=doWhatYouWant(line)
            f.write(newline)
        f.close

Thanks for your help.

You need to add an `r` prefix to the `rootdir` string to prevent the backslashes from being interpreted as escape character. In order to process only CSV files, you could use [my answer](http://stackoverflow.com/a/17283304/355230) to related question except use `'*.csv'` as the pattern instead of the one shown. Also, at the end, you'll need an `f.close()` to actually call the method. — martineau, Jun 25 '13 at 04:17

Timothy · Answer 1 · 2013-06-22T02:10:35.240

3

This code below works. I have commented what were modified inline.

import os

rootdir= 'C:\\Documents\ and\ Settings\\Guest\\My\ Documents\\Code\\' 
#use '\\' in a normal string if you mean to make it be a '\'   
#use '\ ' in a normal string if you mean to make it be a ' '   


def doWhatYouWant(line):
    print line
    return line 
    #let the function return, not only print, to get the value for use as below 


for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        f=open(rootdir+file,'r') #use the absolute URL of the file
        lines = f.readlines()
        f.close()
        f=open(file,'w') #universal mode can only be used with 'r' mode
        for line in lines:
            newline=doWhatYouWant(line)
            f.write(newline)
        f.close()

edited Jun 22 '13 at 02:10

answered Jun 22 '13 at 02:05

Timothy

4,467
5
28
51

Thanks @Skyler I have just tried to implement this but got the following error. Traceback (most recent call last): File "C:/Documents and Settings/Guest/My Documents/SC/Actual work/Part2/New Folder/New Folder/ohno.py", line 13, in f=open(rootdir+file,'r') #use the absolute URL of the file IOError: [Errno 2] No such file or directory: 'C:\\Documents and Settings\\Guest\\My Documents\\SC\\Actual work\\Part2\\New Folder4033_1800_Data.csv' – Newbie Jun 22 '13 at 02:07
that because your URL is too complex. Could you tried to make the URL clean? without space or special char. – Timothy Jun 22 '13 at 02:17
Would it be better if I made the URL C:\Documents and Settings\Guest\My Documents\ABC for example? – Newbie Jun 22 '13 at 02:18
I don't have a windows system to try it out. you might want to just try "C:\\ABC\\" firstly to see what would happen. – Timothy Jun 22 '13 at 02:24
Hmmm @Skyler it does help a little but not really what I am chasing but thanks anyway – Newbie Jun 22 '13 at 02:37
@Kristofer: Just use an `r` [string prefix](http://docs.python.org/2/reference/lexical_analysis.html#grammar-token-stringprefix) to make it a raw string which use different rules for interpreting backslashes, i.e. `r'C:\Documents and Settings\Guest\My Documents\Code'`. – martineau Jun 25 '13 at 03:16
@martineau Yes you are right. I did't tell him it in order to let him know at which char exactly the problem occurs:-) – Timothy Jun 25 '13 at 03:26
@Skyler: Don't understand why you thought that detail was so important to understood as opposed to just making it a non-issue. – martineau Jun 25 '13 at 04:23
@martineau: apprently that the OP doesn't know the knowledge of escape chars. My opinion is that if he takes raw string too early, then he may never know this knowledge. and I think the escape char is a very basic knowledge for a coder, as long as not just think about this specific problem. – Timothy Jun 25 '13 at 20:18

Opening all files in a directory for CSV and reading each line Python

1 Answers1