Taking in multiple .csv files with CSVhelper

Question

I am taking in all .csv files from a directory, taking a few columns from each file using CSVhelper and then writing to a new .csv file. However, when I'm trying to take in the files it seems to only be taking the last file in the directory and I can't understand why. I take the files in like this:

static void Main(string[] args)
    {
        string sourceDirectory = @"C:\Users\SourceDirectory";

        var csvFiles = Directory.EnumerateFiles(sourceDirectory, "*.csv", SearchOption.AllDirectories);

        foreach (string currentFile in csvFiles)
        {
            readFile(currentFile);
        }
    }

And then perform the changes in the files as such:

public static void readFile(string currentFile)
    {
        using (var sr = new StreamReader(currentFile))
        {
            using (var sw = new StreamWriter(@"C:\Users\newFile.csv"))
            {
                var reader = new CsvReader(sr);
                var writer = new CsvWriter(sw);

                //CSVReader will now read the whole file into an enumerable
                IEnumerable dataRecord = reader.GetRecords<dataRecord>().ToList();

                foreach (dataRecord record in dataRecord)
                {
                    //Choose which data values you want to keep
                    writer.WriteField(record.info1);
                    writer.WriteField(record.info2);
                    writer.WriteField(record.info3);

                    //Moves the pointer onto the next record
                    writer.NextRecord();
                }

I can't understand why it's not taking all the files in and even more confused as to why it's only taking the last?

Is it that it is going through all your source files but your code is overwriting your output file each time, so at the end it only contains the output from the last one. — Andy, May 05 '17 at 19:31

score 2 · Accepted Answer · answered May 05 '17 at 19:32

2

The docs for the StreamWriter constructor that takes only one parameter says

The path parameter can be a file name, including a file on a Universal Naming Convention (UNC) share. If the file exists, it is overwritten; otherwise, a new file is created.

So, at each loop you overwrite the previous C:\Users\newFile.csv with the content of the current source file ending with the content of the last CSV file loaded.

If you want to append to the same file you need the constructor that takes a boolean to append

using (var sw = new StreamWriter(@"C:\Users\newFile.csv", true))

answered May 05 '17 at 19:32

Steve

213,761
22
232
286

1

Perfect, thank you!! – johnfish92 May 05 '17 at 19:38
Hope you don't mind if i ask a question. The .csv file i am reading in has 7 headers, however the one i am writing to i want to have 11 headers, some which are different than my class dataRecord. Is this possible with CSVHelper? It throws me an error saying that the header isn't in the class and then if i add it to the class it says it isn't in the file it is reading in.. which obviously it's not going to be. – johnfish92 May 05 '17 at 20:10
Sorry, but I never used the CSVHelper library. Probably your best chance is to post a new question detailing the problem – Steve May 05 '17 at 20:18

Taking in multiple .csv files with CSVhelper

1 Answers1