Stack Overflow
Stack Overflow

Questions tagged [rocksdb]

RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads.

About

RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads.

RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation.

Links

474 questions
6
votes
2 answers

What is the effect of number of levels in levelled compaction?

I know how levelled compaction works in DBS like Cassandra, rocksdb etc. Some have max number of levels 4 and some have 7. How does this number affect compaction process? Why can't I have just 2 levels, 1st one which has flushed mem-table data…
Bishnu
  • 383
  • 4
  • 14
6
votes
0 answers

Limit Memory Usage of Kafka Streams

Is there a way to limit or define the max memory usage of a kafka streams application? I have enabled caching with my state stores but when I deploy in Openshift I get OOM killed on my pods. I have checked I have no memory leakes and all my state…
Chris
  • 1,299
  • 3
  • 18
  • 34
6
votes
1 answer

Why kafka streams state dir is in /tmp/kafka-streams?

I'm not sure if it's already answered. As I didn't get proper explanation, posting my question here. Why kafka streams state.dir is stored under /tmp/kafka-streams? I know I can change the path by providing the state dir config in the stream code…
kuti
  • 161
  • 1
  • 3
  • 13
6
votes
1 answer

Kafka KStream Related Message Events in Sliding Window

We have a situation in which I think Kafka Streams could help, but I cannot find any documentation or examples that show how. There is one similar question I found, but it does not have any implementation advice: Kafka Streams wait function with…
mikemikemike
  • 61
  • 3
6
votes
1 answer

MongoDB server freeze - large amount of collections

We have large MongoDB database (about 1,4mln collections), MongoDB 3.0, engine rocksDB, operating system Ubuntu 14.04. This DB is located on virtual machine (VmWare vCloud) with 16 cores and 108 GB RAM (currently mongoDB used 70GB of memory without…
Kenny6
  • 116
  • 6
6
votes
3 answers

RocksDB IO Error Lock: No locks available

I'm using RocksDB on iOS and am using an AsyncStorage adapter along with redux-persist. Whenever I launch the app I get the error: Failed to open db at path…
CWitty
  • 4,488
  • 3
  • 23
  • 40
6
votes
1 answer

Fastest persistent key/value db for fixed size keys and only insert/fetch (no delete/update)?

Given the following requirements of a persistent key/value store: Only fetch, insert, and full iteration of all values (for exports) are required No deleting values or updating values Keys are always the same size Code embedded in the host…
Vinnie Falco
  • 5,173
  • 28
  • 43
5
votes
1 answer

Why LevelDB and RocksDB need a `masked CRC32`

From the crc32.h of leveldb or rocksdb, we can find a comment saying: static const uint32_t kMaskDelta = 0xa282ead8ul; // Return a masked representation of crc. // // Motivation: it is problematic to compute the CRC of a string that // contains…
Myrfy
  • 575
  • 4
  • 11
5
votes
1 answer

Why enable Record Caches In Kafka Streams Processor API if RocksDB is buffered in memory?

I am trying to understand RocksDB behavior in Kafka streams processor API. I am configuring a persistent StateStore using the default RocksDB that KStreams provide. StoreBuilder countStoreBuilder = Stores.keyValueStoreBuilder( …
iah10
  • 157
  • 3
  • 12
5
votes
1 answer

Kafka Streams Window Store Retain Duplicates

The docs are unclear. When would I want to set retain duplicates to false/true. What is this used for? Is it for something particular in…
Chris
  • 1,299
  • 3
  • 18
  • 34
5
votes
0 answers

How do I resolve a failure to compile librocksdb-sys error on RHEL 7?

I have been trying to build the Canvas Data Loader. I've gone as far as step 10 and it's been pretty easy for me to resolve dependencies on a cargo build fail, but I've been stuck trying to resolve the librocksdb-sys v5.14.2 dependency for the past…
Latifat Braimah
  • 51
  • 4
5
votes
1 answer

RocksDB: static library size that's built from source is really large

When I build RocksDB using make static_lib that produces a 200MB+ librocksdb.a file, but when I install the same version through a package manager (compared to both Brew and apt), the .a file is only about 11MB. What am I missing? Size of the…
jeffreyveon
  • 13,400
  • 18
  • 79
  • 129
5
votes
1 answer

How to delete data older than some days in rocksdb or leveldb ?

I want to create a tool to delete all of the data older than some days in rocksdb or leveldb, but i don't know how to start it.
Michael
  • 367
  • 5
  • 12
4
votes
1 answer

Kafka Streams groupByKey not working due to RocksDB in Mac m1

I'm getting the below rocksdb error when I try to use groupByKey feature of kafka streams in a function. whereas a simple consumer function works fine. Env: confluent v1.30.0 (confluent running on single-node development environment) OS: Apple m1…
Saravanakrishnan Pk
  • 451
  • 1
  • 5
  • 15
4
votes
1 answer

Apache Flink checkpointing stuck

we are running a job that has a ListState of between 300GB and 400GB and sometimes the list can grow to few thousands. In our use case, every item must have its own TTL, therefore we we create a new Timer for every new item of this ListState with a…
Rocel
  • 1,029
  • 1
  • 7
  • 22
1
2
3
…
31 32