LSM Tree has been found successful use in many no-sql engines, its data are sorted by keys not like hashing tables thus enabling many potential use beyond a kv store. For example, a time series database (TSDB) may be a good fit using level db as its engine. How about traditional RDBMS and many table systems? Is LSM-tree like data engines are a good fit as well?
Asked
Active
Viewed 824 times
1 Answers
1
It might be. If you're going to design indexes in a way to exploit strengths of leveldb (namely fast sequential reading) then it might work well.
In fact I have build small relational database on top of leveldb (linqdb) where index is just sorted values of column stored as key-values. My findings are that querying such structure isn't as fast as sqlite's indexed columns (about 40% slower) but writing outperforms by a big margin.
Of course there are many factors in query speed, LSM is just an underlying data structure which best shines at writing.
Additional info here
ren
- 3,843
- 9
- 50
- 95
-
In fact, we are trying to build a table system, but read or batch write may be the typical use case. Currently we are using a in memory hash index, but that doesn't support range query and sorted query. I am looking at LSM because it doesn't cost that much of memory and its keys are stored ordered. – bugs king Aug 03 '16 at 14:33
-
@bugs king The LSM is best fit for disk (large chunks of data written as in merge sort), so not sure if its the best thing for in memory index – ren Aug 03 '16 at 14:50
-
I found there is actually a lsm-tree based mysql in development which is called myrocks, but there are too limited resource about their performance and insight of latency. – bugs king Aug 04 '16 at 04:25
-
comparing read performance with sqlite over a small database is not fair. sqlite is always faster over small datasets. – amirouche Aug 26 '16 at 09:07
-
1@amirouche I've compared on ~20GB set which isn't exactly small. But the result probably is due to many factors (LSM vs B-tree being one of them) – ren Aug 26 '16 at 09:50