Parsing CSV from HttpResponseMessage using CSVHelper

Question

Currently I am using a two step approach to fetch data from the Web Api and deserialize the CSV records into objects.

var response = await httpClient.GetAsync(queryString);
using (var reader = new StreamReader(await response.Content.ReadAsStreamAsync()))
{
    var csvr = new CsvReader(reader);
    var responseValue = csvr.GetRecords<TrainingDataSetData>().ToList();
    result.Readings.AddRange(responseValue);
}

How do I optimize this code?

Looks like, the code is reading the response two times. ReadAsStreamAsync and GetRecords. What I really need is var responseValue = csvr.GetRecords().ToList(); Only reason I am using the ReadAsStreamAsync is because CSVReader's StreamReader ctor arg. — jayanand, Aug 20 '18 at 19:15

georgevanburgh · Accepted Answer · 2018-08-23T12:14:14.700

If you're trying to avoid creating an intermediate MemoryStream - you could use the GetStreamAsync method on HttpClient, which should return the raw NetworkStream for you pass straight into CsvHelper, instead of ReadAsStreamAsync, which will default to reading the full response into a MemoryStream before returning.

using (var reader = new StreamReader(await httpClient.GetStreamAsync(queryString)))
{
    var csvr = new CsvReader(reader);
    var responseValue = csvr.GetRecords<TrainingDataSetData>().ToList();
    result.Readings.AddRange(responseValue);
}

If you still need access to the HttpResponseMessage, you could achieve the same effect by using HttpCompletionOption.ResponseHeadersRead, which won't buffer the response before returning.

var response = await httpClient.GetAsync(queryString, HttpCompletionOption.ResponseHeadersRead);

As to whether or not this is actually more efficient - this is something that would require benchmarking in your particular environment to decide, since it may be conditional on the size of the response, speed of the network etc.

Parsing CSV from HttpResponseMessage using CSVHelper

1 Answers1