We have a pretty memory heavy OData/Web API written in C# and running on an Azure App Service instance. Some specific endpoints which get used rarely can take a long time to execute (20s) or they can take an extremely long time (10min) depending on the query. We would like in that case to have some of sort of background task mechanism for executing these kind of tasks, ideally a header we can attach to any call to tell the API to run it in the background. We have tried:
- Returning the request immediately with a tracking id and then running the SendAsync task in a HostingEnvironment.QueueBackgroundWorkItem, however when the work is completed it throws an exception as its trying to do something with the initial web request which is already disposed
- Returning the request immediately with a tracking id and then passing the SendAsync to HostingEnvironment.QueueBackgroundWorkItem as a function, however that also throws an error upon activating
- Returning the request immediately with a tracking id and then enqueuing a job in the Hangfire which will make a direct call to the app service and can wait as long as it takes. However even using the internal dns, the azure app service has a timeout of 230 seconds, and we can not change that in any way
- We've also tried running the business code itself in a different thread, but we use autofac DI and if we try and get a new scope it still closes when the original request returns.
Similar SO questions have gotten answers that I don't think apply to us:
- Running the work itself in Hangfire, that would mean spinning up the whole API again, just for these rare calls, also we can't do that because our Hangfire is on AKS on Linux, and the API is .net framework.
- Using a webjob - similar issue as the api is very complex and deploying it to a webjob seems very dangerous, also we will then have to pay for 24/7 webjob.
- Using a queue, the problem is that I don't really know how we can take a message from a queue and then execute it against the odata controllers, which we will need to do to populate various contexts and then get data from the DB that we need for these request.
- Using IHostedService - Our API is .net framework and not .net core
Basically any thoughts on how we can achieve this, running an odata web request for as long as needed after we return a tracking id to the client (we understand the risks), without having to do another deployment of our API?
