Stack Overflow
Stack Overflow

Questions tagged [rocksdb]

RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads.

About

RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads.

RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation.

Links

474 questions
3
votes
1 answer

Combine multiple Rocksdb databases

There is a use case for which I have to read huge Parquet file and convert into Rocksdb binary, So I decided to use spark (because everybody is familiar with it in my team). And from Rocksdb side I know it's not distributed and you can not…
Kaushal
  • 3,237
  • 3
  • 29
  • 48
3
votes
2 answers

RocksDb: Multiple values per key (c++)

RocksDb: Multiple values per key (c++) what i am trying to do I am trying to adapt my simple blockchain implementation to save the blockchain to the hard drive periodically and so i looked info different db solutions. i decided to use RocksDb due…
Ariel Hurdle
  • 68
  • 1
  • 13
3
votes
0 answers

Kafka Streams State Store Unrecoverable from Change Log Topic

When our kafka stream application attempts to recover state from the changelog topic our rocksdb state store directory continually grows (10GB+) until we run out of disk space and never actually recovers. How I can reproduce. I start up our…
Chris
  • 1,299
  • 3
  • 18
  • 34
3
votes
0 answers

Kafka Stream memory management (Ktable, RocksDb)

Hi I do not seem to be able to correcly scale my pod for a Kafka stream application (running on java 11 jre) and keep on having OOMKilled containers. kafka stream topology The job consists in an Aggregation of quite a lot of concurrent values I use…
Antonin
  • 879
  • 2
  • 10
  • 27
3
votes
1 answer

In rocksdb Is there any way to know upto what size a file can go in a level?

I am working on rocksdb,but unable to get an option which can tell me the maximum limit of size file inside a level ? And if once it reaches that maximum size how files gets split in RocksDB?
Rishav
  • 31
  • 2
3
votes
2 answers

Embedded database with concurrent read and write

I am looking at a database solution for a concurrent writer and reader problem. I need an embedded database which will be written by a single writer process. The same database will be read by a single reader process and these processes run…
eager
  • 53
  • 1
  • 5
3
votes
1 answer

Flink RocksDB Performance issues

I have a flink job (scala) that is basically reading from a kafka-topic (1.0), aggregating data (1 minute event time tumbling window, using a fold function, which I know is deprecated, but is easier to implement than an aggregate function), and…
OmriManor
  • 255
  • 4
  • 14
3
votes
1 answer

How to get the storage space of a table in the MariaDB MyRocks storage engine

I'm trying to get the space that a table using the MyRocks storage engine takes on disk and I'm getting inconsistent results between what I expect, a query on information_schema and the size as reported by the OS. To get the size as reported by…
Cydrick Trudel
  • 9,957
  • 8
  • 41
  • 63
3
votes
1 answer

How to refer to a particular column family in put and get in rocksdb with java client?

Consider the following code try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions().optimizeUniversalStyleCompaction()) { final List cfDescriptors = Arrays.asList( new…
Jal
  • 2,174
  • 1
  • 18
  • 37
3
votes
1 answer

Why Bloom filters cannot handle range queries?

Context: I'm reading about RocksDB and LSM trees, from my understanding Bloom filter is used to avoid multiple I/Os for item retrieval in all the storage levels. And I'm ok with that. Apparently, one of the challenges is that Bloom filter cannot be…
Murray
  • 97
  • 8
3
votes
1 answer

How long does KTable data maintain in Kafka? Or how can I control the time

I know Kafka uses RocksDb internally. If I use KTable to store data dictionary, will it be deleted after a specific time? If that so, how can I make it last unlimited time?
Archon
  • 1,385
  • 1
  • 15
  • 30
3
votes
3 answers

Embedded key-value db vs. just storing one file per key?

I'm confused about the advantage of embedded key-value databases over the naive solution of just storing one file on disk per key. For example, databases like RocksDB, Badger, SQLite use fancy data structures like B+ trees and LSMs but seem to get…
rampatowl
  • 1,722
  • 1
  • 17
  • 38
3
votes
0 answers

rocksdb how to disable log when OpenForReadOnly?

I open the db using OpenForReadOnly and I want it really just do readonly. But It actually create a lot of log files. How to disable it. ::rocksdb::DB* result; ::rocksdb::Options options; options.create_if_missing = false; options.keep_log_file_num…
J. Doe
  • 632
  • 1
  • 4
  • 11
3
votes
2 answers

RocksDB vs Cassandra

Both MyRocks (MySql) and Cassandra uses LSM architecture to store their data. So I have populated around 5 million rows in MySql with MyRocks as storage engine and also in Cassandra. In Cassandra it takes only 1.7 GB of disk space while in MySql…
Aravind
  • 163
  • 2
  • 13
3
votes
1 answer

rocksdb.errors.RocksIOError: IO error: While lock file: sample.db/LOCK: Resource temporarily unavailable

How to remove the LOCK on rocksDB I try to run the following code but getting the following error * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) * Restarting with stat Traceback (most recent call last): File "hello flask.py", line 18,…
Logic
  • 2,230
  • 2
  • 24
  • 41
1 2 3
…
31 32