When I start my Kafka Streams application many directories and files are created under a temp rocksdb directory. Can I reduce the amount of disk space consumed by RocksDB? The total usage seems to be greater than the space needed by the corresponding Kafka store. Someone said it could be related to compaction or rolling. Can I control the compaction or rolling to reduce disk usage?
Asked
Active
Viewed 2,157 times
2
-
Try tuning `write_buffer_size` option of Rocksdb. – lovesh Jun 17 '18 at 23:46
1 Answers
1
You can pass in RocksDB configs with CustomRocksDBConfig class
https://docs.confluent.io/current/streams/developer-guide/config-streams.html#rocksdb-config-setter
Matthias J. Sax
- 59,682
- 7
- 117
- 137
-
with general documentation on rocksdb here: https://github.com/facebook/rocksdb/wiki and the options best documented here: https://github.com/facebook/rocksdb/blob/master/include/rocksdb/options.h and here: https://github.com/facebook/rocksdb/blob/master/include/rocksdb/advanced_options.h – midor Mar 22 '18 at 12:25