2

This is my first post so please bear with me if I've missed any guidelines.

I'm implementing a solution for throttling keys on the fly, and it revolves around a dictionary that has to be locked before every request (as multiple threads can ask for and modify the same key at the same time). As expected the number of items in the dictionary does not matter but the number of threads being used increases lock contention.

If I can't get a lock on the dictionary I just return OK, so as to not delay the caller. Each successfully taken lock takes around 5us, each unsuccessful about 600uS. So not only does lock contention cause a potentially throttled key to be allowed to proceed, it also takes a lot of time. Locking mechanism is using Monitor.TryEnter.

Test parameters: 20k items in dictionary, equal weighted

With 10 threads I get 140k tps, with 6% lock contention With 20 threads I get 90k tps, with 10% lock contention With 30 threads I get 80k tps, with 14% lock contention

While these are ridiculously high TPS numbers that will likely never be hit on a single box, the problem still stands when the number of accessing threads is high, even with low TPS.

What would the best way to improve this situation be? Perhaps I can have a sharding mechanism with individual locks for multiple dictionaries, and do a separate lookup to find which dictionary to use?

Thanks :)

ArvN
  • 21
  • 3
  • Does [ConcurrentDictionary](https://msdn.microsoft.com/en-us/library/dd287191(v=vs.110).aspx) not work? – Dan Wilson Apr 18 '18 at 17:33
  • It's a good point, but as I understand it concurrent dictionary could have the item when checking for existence, and then not have it when I access it if modified by a different thread (very possible in my case), so I will still need locking. Additionally I found ConcurrentDictionary to be around 3x slower than a dictionary locking approach in my tests, especially when the same keys are being asked for and modified 100s of times a second. Every access to the dictionary results in either an add/update or remove from the dictionary, so the access pattern is 50-50 read/write. – ArvN Apr 18 '18 at 17:43
  • Actually I think I'm wrong on the lock with ConcurrentDictionary, the point of TryAdd and TryRemove is to avoid the lock. However it still performs much worse in Add/Remove scenarios, and around equivalent in read (although you'd even expect it to be faster). – ArvN Apr 18 '18 at 18:17
  • I wouldn't expect it to be as fast as handling the locks for a specific situation; I would just expect it to be thread-safe and reliable. I wonder if it's possible to partition the keys between many concurrent dictionaries? – Dan Wilson Apr 18 '18 at 18:28
  • That's something I did consider, sharding the keys between dictionaries might be a good option. The time to shard should be miniscule if anything. Thanks for that suggestion – ArvN Apr 19 '18 at 18:29
  • One other thing that I'm doing that might help is instead of taking a lock on the dictionary for the entire processing of the item, I only need to take a lock when adding to the dictionary. Readers should not need to lock. – ArvN Apr 19 '18 at 18:31

0 Answers0