1

I have an WebAPI project with several endpoints. One (of several) endpoint calls an external endpoint via a static HttpClient, caches the result and answers following requests for a specified amount from the cache.

We sometimes have the situation were the cache is invalidated, 200 requests / second arrive at the endpoint and the external endpoint is very slow or unresponsive. The first request to the external endpoint sees that the cache is empty and queries the external endpoint. All subsequent requests also start an external HTTP GET call via the HttpClient since the first request did not write the result into the IMemoryCache. Basically, we further create load on the external endpoint if it's unresponsive.

I'm trying to find a way that our WebAPI Method only fires one external HTTP GET call and all other requests wait for the same answer. I've read answers like How to Throttle all outgoing asynchronous calls to HttpClient across multiple threads in .net Core API project to throttle requests but I want a different kind of throttling for one endpoint with one global HttpClient instance

citronas
  • 19,035
  • 27
  • 96
  • 164
  • 4
    Use a Singleton or background service to refresh the cache instead of allowing every endpoint to initiate a GET. This way you can eliminate concurrent calls. Perhaps even hide the cache itself behind a service that returns a value or initiates a refresh – Panagiotis Kanavos Oct 14 '21 at 09:05
  • 3
    Another possibility would be to *decouple* cache lookup from refreshing. Allow endpoints to retrieve slightly stale values and trigger a refresh in the background. – Panagiotis Kanavos Oct 14 '21 at 09:12
  • 1
    @PanagiotisKanavos Great ideas, thank you! You can post them as answers – citronas Oct 14 '21 at 09:25

0 Answers0