0

I have a WEB API which returns some data which includes large byte array [file size is 400 MB] Now i am getting a response from API but while putting it into my class, i am getting an error as -

Exception of type 'System.OutOfMemoryException' was thrown at newtonsoft.json.JsonTextReader.PreparebufferForReadData

While calling an API -

 HttpResponseMessage response = httpClient.GetAsync($"api/getdoc/{Id}").Result;

Here i am getting the valid response but it breaks down in ReadAsAsync

if (response.IsSuccessStatusCode)
{
    responseCollection = response.Content.ReadAsAsync<IList<DocServiceResponse>>().Result;
 }

My DocServiceResponse class -

   public sealed class DocServiceResponse
{
    #region Constructors

    public DocServiceResponse()
    {
    }

    #endregion

    #region Properties

    public Guid Code { get; set; }
    
    public byte[] File_Stream { get; set; }

    #endregion
}

Seems like its failing while deserializing byte array. Is there way to do it?

omkar patade
  • 1,442
  • 9
  • 34
  • 66
  • 6
    `.Result;` is a bug-bug, but not as bad as `which includes large byte array [file size is 400 MB]`. Don't do that. If you want to return a file, return a file. Don't convert the file into a string. taking *5 times as much space*. Each byte converted into a hex value (eg 0xFF) takes 4 bytes for the hex encoding and 1 for the separator. Just return the file, and have the client read the stream – Panagiotis Kanavos Mar 08 '21 at 19:05
  • Isn't byte array the return type we have to use when sending a file through web API ? – Arcord Mar 08 '21 at 19:08
  • @Arcord files aren't send as anything other than files. There's no reason to convert them to JSON. When you download a ZIP or executable from a site you get a stream of bytes with the proper headers, not a JSON with the entire file as string in the response – Panagiotis Kanavos Mar 08 '21 at 19:09
  • Of course not in JSON but when you return a file from a controller, isn't the return type "Byte[]" (except if you keep the generic ActionResult of course)? – Arcord Mar 08 '21 at 19:10
  • @Arcord When you return binary data eg through `return File()` the client gets a stream of bytes, that can be read with `GetByteArrayAsync()` if the array is small, or `GetStreamAsync`. All results types, including `FileResult` inherit from the abstract [ActionResult](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.mvc.actionresult?view=aspnetcore-5.0) – Panagiotis Kanavos Mar 08 '21 at 19:14
  • @PanagiotisKanavos i am not i have option here to return the file itself as we are using FileTable of SQL which stores actual content of files in File_stream column which has type as varcharbinary(max). So we are taking all the data and converting into byte array and returning to client which finally converts in into CSV. This works perfectly if the file size is less than 400 but breaks down if it goes beyond 400 – omkar patade Mar 08 '21 at 19:47
  • @omkarpatade `This works perfectly` no it doesn't. It's horribly slow an eats up a huge amount of server and client RAM. The customer/upper management just hadn't noticed how much they were paying yet. `we are using FileTable of SQL which stores actual content of files in File_stream` all the more reason to return the data as a stream, not as bytes serialized as JSON. You already have a FileStream with the data, just return it to the client. ASP.NET Core will read the data from the FileStream and send it to the client – Panagiotis Kanavos Mar 08 '21 at 20:30
  • @omkarpatade `So we are taking all the data and converting into byte array` don't do that. The very reason `FileTable` exists is so developers *don't* have to do that. Otherwise, why not just load the entire blob with a single query? – Panagiotis Kanavos Mar 08 '21 at 20:33
  • @omkarpatade `which finally converts in into CSV` what's the point of converting a single cell result into CSV? Unless someone stored a CSV as a `varbinary(max)`. Even then, you could just return the file stream with a `text/csv` content type. The client would get a CSV without having to convert anything. – Panagiotis Kanavos Mar 08 '21 at 20:37
  • @omkarpatade which stack are you using? ASP.NET MVC? Web API? ASP.NET Core? You could create a custom ActionResult, perhaps inheriting from [FileStreamResult](https://github.com/aspnet/AspNetWebStack/blob/master/src/System.Web.Mvc/FileStreamResult.cs) that would stream directly from the database *and* close the connection when done. – Panagiotis Kanavos Mar 08 '21 at 20:44
  • @PanagiotisKanavos Thank you for your response. Its a WEB API which is storing/retriving the file data which is present in FileTable. Various MVC applciations/SSIS packages are using this API to store different types of file like CSV,DOC,JPG. They are converting these files into byte array and passing it API which stores it in varcharbinary format. Now changing the complete sturcture looks impossible here – omkar patade Mar 09 '21 at 10:59

0 Answers0