0

I am using Azure Functions, and have stumbled upon an issue.
When requesting large amount of data from an external source, it seems the stream is shut. I have simplified the example as simple as I can below it will silently fail, and return 500. By silent I mean there are no errors in Application Insights that I can see, and no way to know the issue.

This works locally as an azure function.

Is there some sort of limit on data that can be read? It doesn't take much memory (70mb or so locally). So really banging my head against a wall the last two days on this one. Any help appreciated!

[FunctionName("FeedDownloads")]
public HttpResponseMessage Run([HttpTrigger(AuthorizationLevel.Anonymous, "get" )]HttpRequestMessage req,
  ILogger log)//, [FromQuery]string format = "", [FromQuery]bool debug = false)
{
  System.Net.HttpWebRequest webRequest = (System.Net.HttpWebRequest)System.Net.WebRequest.Create("{large 1.5GB gzipped file}");
  webRequest.AutomaticDecompression = System.Net.DecompressionMethods.GZip;
  var webRequestResponse = webRequest.GetResponse();

  var res = req.CreateResponse(HttpStatusCode.OK);
  res.Content = new StreamContent(webRequestResponse.GetResponseStream());
  res.Content.Headers.ContentType = new MediaTypeHeaderValue(webRequestResponse.ContentType);
  return res;
}
tank104
  • 323
  • 4
  • 15
  • Just to be clear the file is 69mb (gzipped) but 1.5gb uncompressed. However the total network ingress should be around 69mb. Also the response is HTTP 500 but OK (which is odd in itself) – tank104 May 01 '20 at 05:31

1 Answers1

0

Azure Functions are not meant for long term communication with the client devices. Large, long-running functions can cause unexpected timeout issues. There are several other problems that you must take a note of before anything. And this is an excerpt from official documentation, functions-best-practices . Functions should be stateless and idempotent if possible. Associate any required state information with your data. For example, an order being processed would likely have an associated state member. A function could process an order based on that state while the function itself remains stateless.

Richard Lee
  • 33
  • 5
  • The function above is stateless though, and takes well short of the 5 minutes that azure funtions run for by default. – tank104 Jun 25 '20 at 22:20