0

I am currently using Lists,Dictionaries and DataTables to store all my data, But I was wondering if there are better data structures or methods(on a lower level) of storing data in order to use memory efficiently.

Any help will be greatly appreciated.

Dreamer78692
  • 215
  • 5
  • 20
  • My biggest worry is long strings. – Dreamer78692 Oct 29 '12 at 06:24
  • 3
    and... Depends on the intended use of the data. For e.g. some containers use more memory for the purpose of allowing quick access to the data or fast insertion etc... – mjv Oct 29 '12 at 06:24
  • I see, its a trade off of space for time.... I think I will have to spend more time designing it before jumping into code thanks. – Dreamer78692 Oct 29 '12 at 06:30
  • 2
    A bloom filter may come in handy if you just need to search/compare existence of some data. – anthonyms Oct 29 '12 at 09:47

1 Answers1

1

It depends on your data and usage. If you want only to store data than the most efficient way is to use array for integers or floats(or any other plain old data) and string pools for strings.

If you need to index data, search by key for example, than one of the most efficient data-structures in terms of size is tries. It's doesn't matter what key type do you use - integer, float or string, trie can be used to create index. Integer or any other keys can be represented as binary strings and inserted into trie. There is plenty of different trie data-structures, that use some kind of compaction to store data more efficiently, for example - Array Mapped Trie. You can also add some compression at lowest level, for example using base 128 coding with integers, or Golomb coding.

Evgeny Lazin
  • 9,193
  • 6
  • 47
  • 83
  • 1
    Isn't searching in a binary heap O(N)? – Justin Oct 29 '12 at 12:16
  • No, it's O(log N) and this is the reason why heap-sort is O(N log N). – Evgeny Lazin Oct 29 '12 at 13:42
  • I guess that's true for heap sort but you are essentially destroying the heap when you transfer the elements from the heap to the sorted array. If you want to do multiple "searches" in the same heap aren't you essentially doing O(log N) to heap-up the desired element then you'd have to add those elements removed back into the heap? – Justin Oct 29 '12 at 13:51
  • Why elements need to be removed from the heap? Once created, binary heap can be queried without modification. – Evgeny Lazin Oct 29 '12 at 14:09
  • That's the way heap-sort works. Build a heap, remove the smallest element N times. You can only assume two things in a (min) heap and that's the smallest element is at the root and the parent is smaller (or equal) than it's children. http://stackoverflow.com/questions/2372994/search-an-element-in-a-heap – Justin Oct 29 '12 at 14:31
  • Sorry, I think about a heap as a way to pack binary search tree to the array. And even then binary search tree can be stored in the array like a binary heap, this is not a binary heap and it can't be queried like a tree in O(log N) time. Thanks for correction. – Evgeny Lazin Oct 29 '12 at 14:42
  • I am not arguing that a heap isn't an efficient storage data structure because it is but it doesn't have O(log n) search. A complete and balanced binary search tree can be packed pretty tight and give you O(log n) search but a heap can always be packed tight. – Justin Oct 29 '12 at 14:55