Why CSVReader only read few files?

Question

I have 22 .csv files that I want to read and write into 1 .csv file This is my internal class

 internal class Record
{
    [Name("RptDt")]
    public string Date { get; set; }
    [Name("Entity")]
    public string Entity { get; set; }
    [Name("ProdFamily")]
    public string ProdFamily { get; set; }
    [Name("ProdGroup")]
    public string ProdGroup { get; set; }
    [Name("ProdType1")]
    public string ProdType1 { get; set; }
    [Name("ProdTypo")]
    public string ProdTypo { get; set; }
    [Name("ProdType")]
    public string Buy { get; set; }
    [Name("Principal")]
    public string Principal { get; set; }
}

This is the write and read code

string[] files = Directory.GetFiles(fbd.SelectedPath, "*.csv", SearchOption.AllDirectories);
string numberFile = files.Length.ToString();                                        
            using (var writer = new StreamWriter(SaveTxt.Text + "\\Result_" + MonthCB.Text + "_" + YearCB.Text + ".csv"))
            using (var csvOut = new CsvWriter(writer, CultureInfo.InvariantCulture))
            {
                for (int i = 0; i < Int16.Parse(numberFile); i++)
                {
                    using (var reader = new StreamReader(files[i]))
                    using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
                    {
                        var records = csv.GetRecords<Record>();
                        csvOut.WriteRecords(records);
                    }
                }
            }

However, the code only write data from the first 2 .csv file. How should I solve this problem?

Are you sure that `files` contain paths for all 22 files? And what happens when the loop enters third iteration? And why you call `ToString()` when assigning value to `numberFile` when few lines below you parse it back to short int? What's stopping you from putting a breaking in the loop and seeing what's happening there by yourself? — Prolog, Jul 25 '20 at 10:48
I have tried to run `Console.WriteLine(files[i])` to check if each file is open and it works. It shows the path for each file. I also have tried with other file with fewer column and row and it also works. So, I have been wondering if there is some kind of limit to read-write row or column in .csv. — irham dollah, Jul 25 '20 at 10:52
So if all 22 files are found there is possibility that reading from 20 of 22 files fails silently resulting in no records read. If the `.csv` files are not that big you could try counting the records and displaying the count like `Console.WriteLine($"{i + 1}. Records read = {records.Count()}")`. Put this line after reading the records, inside the for loop. — Prolog, Jul 25 '20 at 11:08
I have use your code as you suggested. I found out only the first file shows the correct no which is 31 lines while the rest is showing a big number which is more than 30000 even though I put only 3 or 4 lines of data in each file. Is it something wrong with my .csv file? — irham dollah, Jul 25 '20 at 14:23
The question is missing debugging details. And you may provide a minimal reproducible example. — aepot, Jul 25 '20 at 16:26

Vivek Nuna · Answer 1 · 2020-07-25T11:09:33.440

0

There are lot of issues in your code, I have tried to fix many of these. Please let me know if still not working.

string[] files = Directory.GetFiles(fbd.SelectedPath, "*.csv", SearchOption.AllDirectories);
using (StreamWriter writer = new StreamWriter(SaveTxt.Text + "\\Result_" + MonthCB.Text + "_" + YearCB.Text + ".csv"))
{
    foreach (string file in files)
    {
        using (var reader = new StreamReader(@"C:\test.csv"))
        {
            while (!reader.EndOfStream)
            {
                writer.WriteLine(reader.ReadLine());
            }
        }
    }
}

Usage of CsvReader can be avoided. You are doing length.ToString() and again converting to int16. This can also be avoided, because length is already int.

edited Jul 25 '20 at 11:09

answered Jul 25 '20 at 10:53

Vivek Nuna

25,472
25
109
197

3

"There are a lot of issues in your code" - if you found issues in OP code, please describe every one of them with explanation why you think it is incorrect, so OP and others can learn and avoid it in the future. Only pasting fixed code does not really help. – Prolog Jul 25 '20 at 11:03
1

I use csvreader because I only want to read certain column only. If I use your code, it will copy everything from the source file. Any idea how I dont need to use csvreader to read and write certain column from source file? – irham dollah Jul 25 '20 at 11:22

score 0 · Accepted Answer · answered Jul 25 '20 at 17:22

0

I have found the answer. I create a new .csv file for each input. Before this I edit from the actual file, so the size become bigger and the line also been counted even the data not exist. Now, it works just fine.

answered Jul 25 '20 at 17:22

irham dollah

169
13

Why CSVReader only read few files?

2 Answers2