How to implement Caching with data size limit?

Question

I have multiple threads asking for data that have to be loaded over network. In order to have less network traffic and faster responses, I'd like to cache data, which are often requested. I also want to limit the Cache's data size.

My class looks something like this:

public class DataProvider
{
    private ConcurrentDictionary<string, byte[]> dataCache;
    private int dataCacheSize;
    private int maxDataCacheSize;
    private object dataCacheSizeLockObj = new object();

    public DataProvider(int maxCacheSize)
    {
        maxDataCacheSize = maxCacheSize;
        dataCache = new ConcurrentDictionary<string,byte[]>();
    }

    public byte[] GetData(string key)
    {
        byte[] retVal;

        if (dataCache.ContainsKey(key))
        {
            retVal = dataCache[key];
        }
        else
        {
            retVal = ... // get data from somewhere else

            if (dataCacheSize + retVal.Length <= maxDataCacheSize)
            {
                lock (dataCacheSizeLockObj)
                {
                    dataCacheSize += retVal.Length;
                }
                dataCache[key] = retVal;
            }
        }
        return retVal;
    }
}

My problem is: how do I make sure, that dataCacheSize always has the correct value? If two threads request the same uncached data at the same time, they will both write their data to the cache, which is no problem, because the data is the same and the second thread will just overwrite the cached data with the same data. But how do I know, if it was overwritten or not to avoid counting its size twice?

It could also happen, that two threads are adding data at the same time resulting in a dataCache size larger than allowed...

Is there an elegant way to accomplish this task without adding complex locking mechanisms?

Instead of trying to "roll you own" caching, take a look at System.Runtime.Caching.MemoryCache. https://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache(v=vs.110).aspx — Kevin, Dec 07 '16 at 15:23
Ok, MemoryCache seems to do the job. I have to check the options for its cache management though. Do you want to answer the question? I can't accept a comment. — Ben, Dec 08 '16 at 10:50

score 1 · Accepted Answer · answered Dec 15 '16 at 16:11

1

Instead of trying to "roll you own" caching, take a look at System.Runtime.Caching.MemoryCache. See comment above.

answered Dec 15 '16 at 16:11

Kevin

1,462
9
9

Vyacheslav Lukianov · Answer 2 · 2016-12-15T20:45:31.223

0

Since you update dataCacheSize inside lock, you can just check here if it would remain correct:

if (dataCacheSize + retVal.Length <= maxDataCacheSize)
{
    lock (dataCacheSizeLockObj)
    {
        if (dataCacheSize + retVal.Length > maxDataCacheSize)
        {
            return retVal;
        }
        dataCacheSize += retVal.Length;
    }
    byte[] oldVal = dataCache.GetOrAdd(key, retVal);
    if (oldVal != retVal)
    {
        // retVal wasn't actually added
        lock (dataCacheSizeLockObj)
        {
            dataCacheSize -= retVal.Length;
        }
    }
}

edited Dec 15 '16 at 20:45

answered Dec 08 '16 at 21:11

Vyacheslav Lukianov

1,913
8
12

This doesn't work. It will still add retVal.Length twice to the dataCacheSize, when 2 threads added the same entry at the same time. – Ben Dec 14 '16 at 10:44
It's possible to use the `GetOrAdd` method instead of `dataCache[key] = retVal` to resolve this race condition (I updated the answer). But it's probably better to use System.Runtime.Caching.MemoryCache. – Vyacheslav Lukianov Dec 15 '16 at 20:49

How to implement Caching with data size limit?

2 Answers2