I would put the question of multiple application instances on the back burner. Not that it doesn't matter, but if you're programming against interfaces then at some point you could replace your implementation with something that's cached.
If you want average request times over a duration like five minutes then you'll need a list that ejects expired entries. Here's a stab at that:
internal class TimestampedEntry<T>
{
internal DateTimeOffset Timestamp { get; private set; }
internal T Value { get; private set; }
internal TimestampedEntry(T value)
{
Timestamp = DateTimeOffset.Now;
Value = value;
}
}
public class ExpiringList<T>
{
private readonly List<TimestampedEntry<T>> _list = new List<TimestampedEntry<T>>();
private readonly TimeSpan _expiration;
public ExpiringList(TimeSpan expiration)
{
_expiration = expiration;
}
public void Add(T item)
{
lock (_list)
{
_list.Add(new TimestampedEntry<T>(item));
}
}
public IReadOnlyCollection<T> Read()
{
var cutoff = DateTimeOffset.Now - _expiration;
TimestampedEntry<T>[] result;
lock (_list)
{
result = _list.Where(item => item.Timestamp > cutoff).ToArray();
_list.Clear();
_list.AddRange(result);
}
return new ReadOnlyCollection<T>(result.Select(item => item.Value).ToList());
}
}
That ensures that when you read from the list it only returns items stored within the specified interval and also deletes the rest. You could create an ExpiringList<TimeSpan>, add the elapsed time for each call, and then inspect the average as needed.
Where to store it? I'd put it in a class with a single instance. That could be a singleton or a static class. I prefer using a dependency injection container that returns a single instance (like Windsor's singleton lifestyle.) I don't like creating singletons. I'd rather create a "normal" class and then manage it to keep a single instance. DI containers like Windsor make that easy.
I think an important factor in an implementation like this is to keep the messy switching logic separate - hidden in some sort of factory as opposed to having an if/then with all the logic to check the average response times and call either API all in one big class.
For example, if you have an interface representing the call to get data, like IMyDataProvider, then you could define a factory like
interface IMyDataProviderFactory
{
IMyDataProvider Create();
}
Your classes just depend on that factory interface. An class that implements IMyDataProviderFactory checks your average response times and returns either the implementation of IMyDataProvider that calls the external API or an implementation that uses your calculation.
That way the complexity of that logic stays separate from whatever classes depend on the APIs.
Windsor is good with those abstract factories too. Other DI containers also make them easy, and this sort of functionality is built into ASP.NET Core. You weren't asking about dependency injection, but I recommend looking into it. It makes it easier to manage this sort of complexity and keep it maintainable.
Going back to multiple application instances and distributed caches - you can see how the factory pattern implementation makes this easier to manage. Let's say today this is one instance but tomorrow you want to share this data via a distributed cache. Where do you make that change? Most of the code that depends on this API won't need to change at all because it doesn't "know" about any of those implementation details. You would change the code that stores the timing of each API call and change the implementation of your factory.
