1

I'm currently using azure functions for the following process :

  • Dowload tar.gz archives from a client ftp. Most of them are daily packages under 20mo, but a few of them are more than 100mo (edited once year ends and replace all previous packages)
  • Extract the archives into blobs
  • Do some stuff later on the blobs

I'm using generic storage blobs (much cheaper than hot blob container). My problem is currently with the archives exceding 100mo : even with 10mins timeout these can't be extracted. I'm wondering if azure functions are the right tool for this or if I should try azure batch? Maybe a mix of both : a function to trigger a batch for the extraction process?

Another solution would be to run the extraction of large archives from my computer which should be more performant than functions VM.

Any ideas on how to do this?

Janusz Nowak
  • 2,595
  • 1
  • 17
  • 36
Pierre Jeannet
  • 255
  • 1
  • 10

1 Answers1

1

If you deploy your Azure Function into an App Service Plan (non-consumption) you can extend the execution limit as needed as then you have some reserved capacity that can handle that. That would likely be easiest as then you don't have to change code/handling based on file size. Similar to writing as a WebJob in App Service Plan. Let me know if you have any questions on setting this up.

jeffhollan
  • 3,139
  • 15
  • 18
  • Well I forgot to mension this solution. But I loose the power of consumption plan to deal with the many files extraction will produce. I also have some doubt on perf, how long would it take to extract a 700mb archive in app service plan? It will be much more expensive too. – Pierre Jeannet Oct 19 '17 at 15:11
  • 1
    Would definitely depend on the size of app service plan on code so hard to say. Other option is you could potentially spin up a container using the Azure Container Instance service to do the long ones. That gives you per-second billing and the "consumption" type pricing - I expect it may be simpler than VM Batch as well. Would let you provision a container to do the unzip and then remove it. – jeffhollan Oct 19 '17 at 16:01