Efficient Int32/Uint32 Sorted Map / Sparse Array

Question

I'm looking for a specialized (and fast) Int32/UInt32 sorted map (that preferably is faster then System.Collections.Generic.SortedDictionary where K is either Int32 or UInt32).

It's going to be used as a sparse array, are there any implementations for .NET?

thr, can you define what "fast" means for your case? SortedDictionary operations are all O(log n), are you looking for some of operations to be O(1)? — Alexei Levenkov, Mar 25 '11 at 15:49
whitout stating the problem more specifically I cannot provide any answer. SortedDictionary is I believe a balanced tree implementation and it has a log n lookup speed. If You use Your collection as mainly bulding and then checking for existence, but then dump the sorted collection somewhere, You can abuse memory impact a bit and build a custom collection to improve lookup speed a bit (store all keys in a dictionary as well), but building it will always cost nlog n — luckyluke, Mar 25 '11 at 15:56
And yes I'm looking for some operations to be as close to O(1) as possible, preferably contains(key) and get(key), iteration should be O(n) — thr, Mar 25 '11 at 15:58
@thr: `SortedList` *can* be faster than `SortedDictionary` depending on the types of operations you're doing. — Powerlord, Mar 25 '11 at 15:58
Create a custom collection that looks up things using dictionary and uses sorting using SortedList (dictionary). . Double the memory fingerprint but since it is sparse array (so not that many elements) it should work pretty well. This is pretty easy and should be pretty performant — luckyluke, Mar 25 '11 at 16:01
I abused in the past (IF the set YOu are using is limited (by let's say milions) I used bit representation. So I had a use case with top element of 100 Milion. I used a BIG array (using unsafe code) of 100/8 bits - 12,5 mil bits = 2 megs and BITWISE operations. So insertion/contains was a frickingly fast bitwise check and sorting was linear time. But it only works with int/uints. Limited in functionality but Permormant as hell an abusing hardcore pointers though, so You need to watch out. But You only need two operations SetBit (simple bitwise operation) and ClearBit (inverse). Regards Luke — luckyluke, Mar 25 '11 at 16:09
If his current implementation is a SortedDictionary, I'm assuming it isn't `SortedDictionary`. I'm assuming he needs to store objects in his collection, so I don't think a bitfield would be enough. — David Yaw, Mar 26 '11 at 19:28

score 1 · Answer 1 · answered Mar 25 '11 at 19:59

As was mentioned in the comments, I'd write a custom collection that uses both a SortedDictionary and a regular Dictionary as its backing store. It doubles your memory usage, but it's the best performance for lookups and iteration. Modifications will be slower, but it sounds like you're mostly interested in fast accesses.

public class DoubleDictionary<TKey, TValue> : IDictionary<TKey, TValue>
{
    private Dictionary<TKey, TValue> backingHash = new Dictionary<TKey, TValue>();
    private SortedDictionary<TKey, TValue> backingTree = new SortedDictionary<TKey, TValue>();

    // For all the modify methods, do it in both.
    // For all retrieval methods, pick one of the backing dictionaries, and just use that one.
    // For example, contains and get on the Hash, iteration on the Tree.
}

Efficient Int32/Uint32 Sorted Map / Sparse Array

1 Answers1