3

The IntValue1 in the csv file is a number without double quotes but when the system reads the row it adds double quotes at the beginning and also double quotes followed by semicolon at the end of the row to delimit the row. Because of this, the IntValue1 starts with a double quotes and the system doesn't recognize it as an int... Please help me to fix this error.

The model:

[DelimitedRecord(",")]
public class MyObject
{
    private int _ID;
    public int ID
    {
        get { return _ID; }
        set { _ID = value; }
    }
    private DateTime _EventDate;
    public DateTime EventDate
    {
        get { return _EventDate; }
        set { _EventDate = value; }
    }
    private string _IPAddress;
    public string IPAddress
    {
        get { return _IPAddress; }
        set { _IPAddress = value; }
    }
}

The code for reading & displaying the data:

private static void GetCsvData()
{
   var engine = new FileHelperEngine<MyTypeObj>();

   //The error occurs at this line:
   var result = engine.ReadFile("C:\\CsvFileName.csv");

   //Code to display the Data
}

The CSV file looks like this:

enter image description here

Content of the row returned by the debugger when the error occurs: enter image description here

Error: enter image description here

Sounouk
  • 95
  • 1
  • 12

2 Answers2

4

Problem #1: Your file is not a valid CSV. The proper way to fix this problem would be to correct the way the file is being exported as it is not exporting a valid CSV file.

This is not a valid CSV file:

"ID,""EventDate"",""IPAddress""";
"1,""2013-01-19"",""11.81.11.00""";
"2,""2012-11-25"",""11.72.41.84""";
"3,""2011-12-27"",""15.80.";"3.36"""
"4,""2014-08-17"",""17.72.";"9.24"""
"5,""2012-01-30"",""90.94.27.108""";
"6,""2013-02-15"",""19.97.27.189""";

The Hack: You can use FileHelpers to import invalid CSV files using the BeforeReadRecord event. Sometimes you are just not in control of what CSV file you may receive for importing.

Your Model:

[IgnoreFirst]
[DelimitedRecord(",")]
public class MyTypeObj
{
    [FieldConverter(ConverterKind.Int32)]
    public int ID;

    [FieldQuoted]
    [FieldConverter(ConverterKind.Date, "yyyy-mm-dd")]
    public DateTime EventDate;

    [FieldQuoted]
    public string IPAddress;
}

Your code:

var engine = new FileHelperEngine<MyTypeObj>();

engine.BeforeReadRecord += (@base, args) =>
{
    // Convert this: "1,""2013-01-19"",""11.81.11.00""";
    //      to this: 1,"2013-01-19","11.81.11.00"
    args.RecordLine = args.RecordLine
        .Substring(1, args.RecordLine.Length - 3)
        .Replace("\"\"", "\"");
};

var result = engine.ReadFile(@"C:\CsvFileName.csv");

You will still have problems with these lines (why is there a semi-colon in the middle of the value???):

"3,""2011-12-27"",""15.80.";"3.36"""
"4,""2014-08-17"",""17.72.";"9.24"""

You could also fix these in BeforeReadRecord, but it would be best to figure out what is going wrong with the export.

tl;dr Fix how the file is exported so you get a VALID CSV. Fixing the import should only be an absolute last resort hack.

joelnet
  • 13,621
  • 5
  • 35
  • 49
  • with this change the system displays a more explicit error: "Error Converting ' "177479' to type: 'Int32'." The value shows a double quote just before 177479. – Sounouk Sep 08 '15 at 18:40
  • @Sounouk You must add [FieldQuoted] to the private int _ID; field – Marcos Meli Sep 08 '15 at 19:50
  • @MarcosMeli I did it, the error after this change is: Error Converting '177479,"2014-01-29","10.91.10.11"' to type: 'Int32'. – Sounouk Sep 08 '15 at 20:31
  • It looks like your datafile might have issues. It is attempting to convert a Date to int. Int is your first field, but you have Date as your 2nd. Look at the file, you will probably notice a date in the first column. You can also try changing your int ID to a string. – joelnet Sep 08 '15 at 22:15
  • @joelnet the problem is that the record starts with this character " and ends with these ones " ; and when I try to get the first field (int) it comes with a double quote... the solution I found for the moment is to use String.Trim(' " ') to remove all the lost double quote, it's not the best practice but it's working :) – Sounouk Sep 09 '15 at 19:11
  • @Sounouk, Can you please post a link to a file that contains some example data. I am not having any problems importing the CSV example that you posted above. – joelnet Sep 09 '15 at 22:09
  • Joelnet is right with the IgnoreFirst comment. ID would have been converted. If you can post a test project with data then Joel or myself can take a look to see what exactly the issue is. The other thing I noticed was that you said the ID field was quoted but it wasn't in the example you gave. – netniV Sep 09 '15 at 22:37
  • @joelnet This line "[FieldConverter(ConverterKind.Int32)]" return this error "Error Converting '"1' to type: 'Int32'." the problem causing that is the character " added by the system to start the row ... I have edited the question to add an image of some fictive data because I can't share the real ones. – Sounouk Sep 10 '15 at 08:59
  • @netniV The problem is not with the type of the ID but with the double quotes that FileHelpers add at the beginning of the row and also at the end followed by a semicolon ... I tried 3 different csv file parser and all of them return the row as I explained before, that's why I used String.Trim(...) to remove manually all these characters... – Sounouk Sep 10 '15 at 10:05
  • FileHekpers should never be adding characters. It may be leaving them in if the field isn't marked as quoted or you change the quote character and it still has the quote character in the source. If you change the ID to a string what do you get in each field? It would be useful to see this per row. – netniV Sep 10 '15 at 10:34
  • 1
    Also, make sure that the quote character is definitely the quote character since extended ascii and Unicode character sets can have similar looking quotes that are not the same. Can you provide a link to a sample project to verify this ? – netniV Sep 10 '15 at 10:36
  • Here is the link to a test project with csv file: https://www.dropbox.com/sh/2jxdr7wju0h7jms/AABYb6amBrb5M-y0_rrjY9p-a?dl=0 – Sounouk Sep 10 '15 at 12:54
  • @Sounouk I have modified my answer to reflect new information in your question. Please review answer :) – joelnet Sep 10 '15 at 17:09
  • @joelnet I did the test with the code you've added and the real data (103990 rows) and everything works great, no bug and your code is very clean. Thank you!!! :) – Sounouk Sep 10 '15 at 19:47
0

Looking at the quick start example on the FileHelpers website, their example csv only contains data - not a descriptive first row of data.

I guess when the FileHelperEngine reads the first item of the first row, which is "ID", it will try to convert it to an integer, which fails...

Dietz
  • 578
  • 5
  • 14
  • No I don't think so because the FileHelperEngine recognizes the first row as names of column and the error occurs as from the second row. – Sounouk Sep 08 '15 at 18:45