1

I have cache implemented and this cache needs to be updated after certain intervals. Creating or updating cache involves calling lot many external APIs and may be little slower. My cache is a simple collection. I have made it static so that all the request refer to same instance. The problem arises when I am updating the cache. Before I put the new data I clear the cache. I simulated this scenario, where in I put some delay just after clearing the data and before updating it with new data. I get the obvious exception of no sequence found. How to handle this scenario. I guess this is very likely situation one would get into in multi threaded application.

I have explored ConcurrentBag, SynchronizedCollection and ReaderWriterLockSlim. Nothing seems to be working (or may be I am not using them rightly) My expectation: The reader thread should use the older version of cache while its being updated and new version right after it

joker22
  • 21
  • 1
  • 1
    Instead of clearing the cache, create a new one, fill it and do an atomic swap. Whoever took the old version will still work with it without a problem. And let the garbage collector clean it up after everyone is done using it. This might be problematic if the cache is huge (like over 1Gb in RAM) though. In such situation you need to pick a thread-safe collection and manually add/delete elements instead of refilling the cache. Which will of course leave a window of inconsistency (often acceptable). Fixing the inconsistency is generally hard, and requires some kind of transactional system. – freakish Apr 26 '23 at 13:53
  • @freakish thanks for the heads up. Have implemented something similar and was able to get close to my requirement – joker22 Apr 30 '23 at 08:21

1 Answers1

0

You have the Interlocked class in the System.Threading which, in turn, has the Exchange generic method. Every time you need you cached to be updated the following procedure it to be invoked:

  1. Collect the data from all the APIs using async/await mechanism; don't forget about the CancellationToken to ensure a reasonable upperbound and use Task.WhenAll heavily to achieve best possible parallelism.
  2. Assemble all the collected (and validated?) data into a new instance of the (assumed) Cache class. It literally means that you have to create a new cache rather than update the current one (which is meanwhile continues to serve to your application as you expected).
  3. You the Exchange<T> method from the above to atomically interchange the old one with the new one; this operation is guaranteed to succeed and to be atomic no matter what, no extra measures have to be taken by your code.

WARNING: the procedure above implies that every your client reads cache only once at a time, basically copies reference to a current Cache instance to a local variable and then talks to it; otherwise you might get into the trouble: client reads the old cache, swap happens, the same client reads (not!) the same cache again and gets entirely inconsistent result.

Zazaeil
  • 3,900
  • 2
  • 14
  • 31
  • Yes after the @freakish I implemented something similar to what you have described . And in addition to it, I used shared ReaderWriterLockSlim to make sure no threads read while its being updated. So reader threads can wait for few milliseconds as I used write lock just to do swap – joker22 Apr 30 '23 at 08:24
  • You don't need the second one. Interlocked will swap the reference atomically. – Zazaeil Apr 30 '23 at 12:27