StringWriter processes strange characters on csv generated

Question

I'm having troubles using StringWriter on our application. I do a rest call over a nosql db and it returns a list of dynamics. I use StringWriter to write a csv file that contains a header and records from my list.

I also tried to extend the StringWriter with a sealed class with constructor method which allows you to enter the type of encoding as a parameter. But trying all the encodings available it still generates wrong charachters.

This is our extension of StringWriter:

public sealed class StringWriterWithEncoding : StringWriter
{
    private readonly Encoding encoding;

    public StringWriterWithEncoding() : this(Encoding.UTF8) { }

    public StringWriterWithEncoding(Encoding encoding)
    {
        this.encoding = encoding;
    }

    public override Encoding Encoding
    {
        get { return encoding; }
    }
}

and this is the code for generate the csv file:

StringWriterWithEncoding sw = new StringWriterWithEncoding();

// Header
sw.WriteLine(string.Format("{0};{1};{2};{3};{4};{5};{6};{7};{8};{9};", "Soddisfazione", "Data Ricerca", "Categorie Cercate", "Id Utente", "Utente", "Categoria", "Id Documento", "Documento", "Id Sessione", "Testo Ricerca"));

foreach (var item in result.modelListDyn)
{
   sw.WriteLine(string.Format("{0};{1};{2};{3};{4};{5};{6};{7};{8};{9};", item.Satisfaction, item.Date, item.Cluster, item.UserId, item.Username, item.Category, item.DocumentId, HttpUtility.HtmlDecode(item.DocumentTitle.ToString()), item.SessionId, 
   item.TextSearch));
}

var response = Request.CreateResponse(HttpStatusCode.OK, sw.ToString());

response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("text/plain");

return response;

When the file is generated on in a column with some text, it display strange chars: Lâ€™indennitÃ di licenziamento del Jobs Act Ã¨ incostituzionale

This is italian, and the wrong chars are seems to be à è ò ' ù etc.

Anyone can suggest a solution? Thank you!

UPDATE

As user suggested, i started using CsvHelper I created a Class and a ClassMap but it still returns corrupted chars.

StringWriter sw = new StringWriter();
CsvWriter cw = new CsvWriter(sw);
using (CsvWriter csv = new CsvWriter(sw))
{
  csv.Configuration.RegisterClassMap<HistorySearchModelCsvHelperMap>();
  csv.Configuration.CultureInfo = CultureInfo.InvariantCulture;
  csv.WriteRecords(csvModelHelperList);
}

Result:

UPDATE 2

The problem is client-side, my action returns the correct text, without broken chars. Action is triggered when i call it with an axios get instance.

axios.get(url, {
   headers: {
      'Accept': 'application/vnd.ms-excel',
      'Content-Type': 'application/vnd.ms-excel'
   }
})
.then(({ data }) => {
   const blob = new Blob([data], {
      type: 'application/vnd.ms-excel',
   });
   // "fileDownload" is 'js-file-download' module.
   fileDownload(blob, 'HistorySearches.csv', 'application/vnd.ms-excel');
   this.setState({ exportLoaded: true, exportLoading: false });
}).catch(() => {
   this.setState({ exportLoaded: false, exportLoading: false });
});

I read to set responseType to blob but even passing the type: 'application/vnd.ms-excel' the chars over my csv file are still corrupted. In my action when i return the Response:

// ... some code

StringWriterWithEncoding sw = new StringWriterWithEncoding();
CsvWriter cw = new CsvWriter(sw);
using (CsvWriter csv = new CsvWriter(sw))
{
    csv.Configuration.RegisterClassMap<HistorySearchModelCsvHelperMap>();
    csv.Configuration.CultureInfo = CultureInfo.InvariantCulture;
    csv.WriteRecords(csvModelHelperList);
}

return Request.CreateResponse(HttpStatusCode.OK, sw.ToString());
// response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/vnd.ms-excel");

return response;

I tried to set content type server-side too, but the format is incorrect anyway.

Are you sure text is "polluted" here? Probably response responses with a different encoding tho, consider searching for stuff like that. — misticos, Oct 04 '19 at 14:37
This is 2019, don't roll your own CSV writer. There's loads of nuget packages, CsvHelper is pretty good. — Neil, Oct 04 '19 at 14:47
Yeah, you right, it's 2019. So I've installed CsvHelper and besides the initial difficulty to configure it, it did not solved my problem. Chars are corrupted yet. — Matteo Pietro Peru, Oct 07 '19 at 08:39
Have you opened with Notepad++? The simple text editor can investigate your encoding problem. Either saving or loading should be wrongly encoded. — Youngjae, Oct 07 '19 at 08:54
Yeah, actually the problem is client-side. I think i must ask a new question. — Matteo Pietro Peru, Oct 07 '19 at 10:03

score 0 · Answer 1 · answered Oct 04 '19 at 14:38

0

If you want to be able to open your csv in Excel, you need to write it with an encoding of Windows-1255.

If you open the csv in a generic text editor and it still displays incorrectly, I'm not sure what's wrong, as your code looks sane.

answered Oct 04 '19 at 14:38

Nick Murphy

129
1
6

score 0 · Answer 2 · answered Nov 25 '19 at 12:02

Solved directly on client-side. I made my own download routine and passed the UTF-8 BOM as first value of response string:

downloadFile2(data, fileName, type="text/string") {
    // Create an invisible A element
    const a = document.createElement("a");
    a.style.display = "none";

    // Using "universal BOM" https://technet.microsoft.com/en-us/2yfce773(v=vs.118)
    const universalBOM = "\uFEFF";
    a.setAttribute('href', 'data:text/csv; charset=utf-8,' + encodeURIComponent(universalBOM+data));
    // Use download attribute to set set desired file name
    a.setAttribute('download', fileName);
    document.body.appendChild(a);
    // Trigger the download by simulating click
    a.click();

    // Cleanup
    window.URL.revokeObjectURL(a.href);
    document.body.removeChild(a);
},

StringWriter processes strange characters on csv generated

2 Answers2