How does yugabytedb achieve high data density?

Question

It is understood that yugabytedb creates n key/value records per SQL row.
Still, how does it manage to achieve high data density?

score 1 · Accepted Answer · answered Jan 14 '20 at 09:18

@AVA

We use DocDB internally which uses a heavily modified RocksDB. A summary from the doc links at the end of comment:

Each table gets split into several tablets. Each tablet being a RocksDB instance. RocksDB instances use size tiered compaction whi
RocksDB instances use size-tiered compaction which has low write IO amplification and higher space amplification (needs 50% free disk space). This is mitigated by having several small tablets. We get low write IO with no space amplification concerns.
Global memstore limits. Big number of tablets won't have overhead in-memory since memstores are used in 1 pool.
Global block-cache will handle caching on all sstables. This further reduces overhead instead of keeping per-rocksdb cache.
Global Throttled compactions and small/big compaction queues will help against compaction storms overwhelming the server
Striping tablet load uniformly across data disks: Each tablet can reside in a different disk (JBOD). This will loadbalance sstables & WAL IO between disks in the machine.
Efficient c++ implemention code. No stop-the-world GC helps with keeping latencies low & consistent.
When the server is at full usage we stop accepting new queries. This will keep the server from crashing or being overwhelmed.
On-disk block compression (low read/write IO).
Uncompressed in-memory block-cache: low cpu overhead on each read & able to server more hot queries.
Disabling Rocksdb WAL and only keeping RAFT wal. This reduces read/write IO.

1 Answers1