How to organize key-values to implement Redis ZSet commands by RocksDB?

Question

I implement Redis ZSet commands by RocksDB, and iterator is really slow with big key, how can I optimize these key-values?

I devide 1 zset key-element-score with 3 RocksDB key-values, such as name2age:linda:25

• meta key:

  "name2age": 1 // count one element, use for zcard

• index key:

  "name2age:linda": 25 // use for zrangebyindex

• score key:

  "name2age:25:linda": NULL // use for zrangebyscore

// using rocksdb iterate to start index, count offset, and select elements
// this will be slow when zset have too many elements

I want a new implementation to rewrite zset, maybe some tips, thanks a lot

Answer 1

To implement a sorted set on top an ordered key-value store, you must rely on probabilistic datastructure called the skip-list.

But in your question the operations that you want to perform do not require a skip list.

See this slides for a intro on skip-lists

And this thread on foundationdb forums on how to implement a leaderboard which is most likely what you want.

This links to a C# implementation that can be found on github

Good luck!