0

We are storing our files into Azure blob, we are giving option to our user to download multiple files in a zip format ( can be 500-1000 files) from our website. We have placed a code but its taking longer time to execute and as we are having our website hosted on Azure Web app its getting 500 request time out. Can some one help us to show how we can improve and make it more faster?

Please find a code that we did.

string[] pods;
                            string fileName = "";
                            try
                            {
                                System.IO.Directory.CreateDirectory(path);
                                foreach (var item in objData)
                                {
                                    pods = item.POD.Split(new string[] { "," }, StringSplitOptions.None);
                                    foreach (var itemPOD in pods)
                                    {
                                        string[] data = itemPOD.Split(new string[] { "/" }, StringSplitOptions.None);
                                        fileName = System.IO.Path.Combine(path, DateTime.Now.ToString("dd_MM_yyyy") + "_" + data[data.Length - 1]);
                                        try
                                        {
                                            var stream = new WebClient().OpenRead(itemPOD);
                                            using (System.Drawing.Image img = System.Drawing.Image.FromStream(stream))
                                            {
                                                img.Save(fileName, System.Drawing.Imaging.ImageFormat.Jpeg);
                                                GC.Collect();
                                            }
                                        }
                                        catch (Exception)
                                        {

                                        }
                                    }
                                }
                                FileDownloads objFileDownload = new FileDownloads();
                                var filesCol = objFileDownload.GetFile(path);
                                using (var memoryStream = new MemoryStream())
                                {
                                    using (var ziparchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
                                    {
                                        for (int i = 0; i < filesCol.Count; i++)
                                        {
                                            ziparchive.CreateEntryFromFile(filesCol[i].FilePath, filesCol[i].FileName);
                                        }
                                    }
                                    return File(memoryStream.ToArray(), "application/zip", "Attachments.zip");
                                }
                            }
                            catch (Exception)
                            {

                            }
Chidrup
  • 103
  • 5
  • 2
    Instead of doing it in web app, can you not do it in some background process? – Gaurav Mantri Dec 28 '16 at 03:24
  • 1
    This should indeed be done using a background process. Also, why don't you parallelize the downloads? And use the async method of `OpenRead`, see https://msdn.microsoft.com/en-us/library/system.net.webclient.openreadasync(v=vs.110).aspx. If you switch to `HttpClient` you can use `async` with `await`. For example with the method `GetStreamAsnc`: https://msdn.microsoft.com/en-us/library/hh551738(v=vs.118).aspx. – Peter Bons Dec 28 '16 at 09:11
  • Yes I was thinking the same to move it in background process but again user should be notified in real time once the zip file is ready to download so might signalR comes into the picture. – Chidrup Dec 29 '16 at 04:52

1 Answers1

0

We are storing our files into Azure blob, we are giving option to our user to download multiple files in a zip format ( can be 500-1000 files) from our website. We have placed a code but its taking longer time to execute and as we are having our website hosted on Azure Web app its getting 500 request time out.

It seems that you get a solution with the help of Gaurav Mantri and Peter Bons. Running long-running tasks in background thread will be better than doing it in your controller action code. According to your description, you host app on Azure web site, Azure WebJob would be a good choice for run your task to zip multiple files. Besides, if you’d like to push notification to client via Signalr, you may need to map users to their connections and retain information about user&connectionid&zipfile, and then you could invoke Signalr hub method from WebJob to push notification to specific user to let him know the zip file is ready.

The following links discussed about “communicate from an Azure WebJob to website with SignalR”, which would be helpful.

  1. Communicate from an Azure WebJob to your website with SignalR
  2. Communication between a WebJob and SignalR Hub
Community
  • 1
  • 1
Fei Han
  • 26,415
  • 1
  • 30
  • 41