1

I am currently working on a project and trying to find out what is the most suitable Azure option to make daily API calls to Dark Sky(https://darksky.net/dev)

In short we are looking to bring weather data inside our data lake storage(Azure Data Lake Store), and we would have to make 100s of thousands of calls a day to dark sky in order to retrieve the data(unfortunatelly they don't provice batch yet). My question is which service would be best to make the calls, get the data and write it to the data lake? The ingestion will not be continous, and I would like to combine the messages in order to minimize write operations on the data lake.

I have been considering Azure functions and Azure batch, but I am unsure which one is more suitable and also more cost efective considering the given requirements.

Thank you for your help.

Cristian Iosub
  • 157
  • 12
  • Probably Azure Functions on an App Service Plan (a dedicated plan as opposed to "Consumption") would be the lowest-maintenance option, – evilSnobu Sep 11 '18 at 09:07
  • B-series VMs are the cheapest option - Watch out for those overprovisioned cores though, https://azure.microsoft.com/en-us/blog/introducing-b-series-our-new-burstable-vm-size/ – evilSnobu Sep 11 '18 at 09:21
  • Thanks for the suggestion. I am also more inclined towards Azure functions, but wanted to hear other opinions since there might be something i'm missing. We are already using functions on a dedicated App Service Plan, and have a VM for that. I will definitely look at the performance and make sure the VM can run everything optimally. Thanks again! – Cristian Iosub Sep 11 '18 at 09:31
  • Azure Container Instances could also be an option if batching is applicable. They're not really designed to run 24/7, i mean they can, they'll just be a little more expensive than a VM - https://azure.microsoft.com/en-us/services/container-instances/. They can stand on their own, AKS is optional, not required. – evilSnobu Sep 11 '18 at 09:43
  • That actually looks like interesting. I guess I missed the release of it. – Cristian Iosub Sep 11 '18 at 11:20

0 Answers0