ConcurrentDictionary's GetOrAdd is not atomic. Any alternatives besides locking?

Question

I'm using concurrent dictionary to hold open files.

To open a new file, I do this:

myDictionary.GetOrAdd (fName, (fn) => new StreamWriter(fn, true));

And with this, I regularly get following exception:

System.IO.IOException: The process cannot access the file '....' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPa
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost)
   at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize)
   at System.IO.StreamWriter..ctor(String path, Boolean append)

To me that means that the operation is not atomic and that the software is trying to open the same file twice.

Is there any other operation I can use instead, which will be atomic? I want to avoid locking because of performance considerations.

M.A. Hanin · Answer 1 · 2014-02-19T00:02:16.257

Since, as Ameen points out, the guarantee is that the key/value pair will only be added once, you can hold a ConcurrentDictionary<string, Lazy<StreamWriter>>, and the value factory should construct the Lazy<StreamWriter> by passing the LazyThreadSafetyMode.ExecutionAndPublication argument (second argument, after the instantiation delegate). This way, you get to limit the lock per-file, without locking the entire dictionary.

private static ConcurrentDictionary<string, Lazy<StreamWriter>> _myDictionary =
    new ConcurrentDictionary<string, Lazy<StreamWriter>>();

public static StreamWriter GetOrCreate(string fName)
{
    return _myDictionary.GetOrAdd(fName,
        new Lazy<StreamWriter>(() => new StreamWriter(fName),
            LazyThreadSafetyMode.ExecutionAndPublication)).Value;
}

wow. sounds really interesting. can you show me an example on how this is done? — Arsen Zahray, Feb 18 '14 at 23:54

score 3 · Answer 2 · answered Feb 18 '14 at 23:35

This is expected. The dictionary itself may be concurrent, because it is providing thread-safe access to the dictionary itself, but you're involving other system resources which are outside the boundaries of a concurrent dictionary. The Concurrent dictionary has no idea what you are doing with the streams.

Might I suggest that instead of using a ConcurrentDictionary in this scenario, you wrap your calls using a ReadWriterLockSlim instance. This gives you read locks (multiple readers) that can also be upgraded to write locks. Therefore when you just need to read, everyone can read at the same time, but in the cases that you need to write - there is a solid lock in place.

Looks useful! I'll definitely try it – Arsen Zahray Feb 18 '14 at 23:38 — Arsen Zahray, Feb 18 '14 at 23:38

score 1 · Answer 3 · answered Feb 18 '14 at 23:42

The value factory method that you pass may be called multiple times, even though only one value will be stored in the dictionary. I can't think of a way to use ConcurrentDictionary to achieve what you want.

This is from the documentation:

If you call GetOrAdd simultaneously on different threads, addValueFactory may be called multiple times, but its key/value pair might not be added to the dictionary for every call.

http://msdn.microsoft.com/en-us/library/ee378677(v=vs.110).aspx

ConcurrentDictionary's GetOrAdd is not atomic. Any alternatives besides locking?

3 Answers3