Stack Overflow
Stack Overflow

Questions tagged [leveldb]

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

LevelDB is a fast key-value storage library written by Google, providing an ordered mapping from string keys to string values.

Inspired by Bigtable, LevelDB is used by many other systems, including Chrome (for IndexDB), Riak (optional back-end), and Bitcoin (blockchain metadata).

Some features

  • Batching writes
  • Forward & backward iteration
  • Compression of the data via Google's Snappy compression library.
  • Data sorted by key; a custom comparison function to override the sort order.
  • Users can create a transient snapshot to get a consistent view of data.

Official Links:

Books

  • Getting Started with LevelDB, By: Andy Dent, Ebook ISBN:978-1-78328-102-2 | ISBN 10:1-78328-102-2
405 questions
6
votes
1 answer

Fastest persistent key/value db for fixed size keys and only insert/fetch (no delete/update)?

Given the following requirements of a persistent key/value store: Only fetch, insert, and full iteration of all values (for exports) are required No deleting values or updating values Keys are always the same size Code embedded in the host…
Vinnie Falco
  • 5,173
  • 28
  • 43
6
votes
1 answer

Scala SBT and JNI library

I am writing a simple app in Scala that uses a leveldb database through the leveldbjni library. My build.sbt file looks like this: name := "Whatever" version := "1.0" scalaVersion := "2.10.2" libraryDependencies ++= Seq( "org.iq80.leveldb" %…
nopper
  • 825
  • 11
  • 18
6
votes
2 answers

Efficient way to check existence in a large set of strings

I have a set of 100+ million strings, each up to 63 characters long. I have lots of disk space and very little memory (512 MB). I need to query for existence alone, and store no additional metadata. My de facto solution is a BDB btree. Are there any…
Drew Sears
  • 12,812
  • 1
  • 32
  • 41
6
votes
5 answers

Anyone used LevelDB store for ActiveMQ message persistence?

At present we are using KahaDB store for message persistence in ActiveMQ and so far good. As per the release notes of ActiveMQ5.6, LevelDB provides enhanced performance. Has anyone tried usign LevelDB and if so could you provide the pros and cons?
srini
  • 83
  • 1
  • 7
5
votes
1 answer

Why LevelDB and RocksDB need a `masked CRC32`

From the crc32.h of leveldb or rocksdb, we can find a comment saying: static const uint32_t kMaskDelta = 0xa282ead8ul; // Return a masked representation of crc. // // Motivation: it is problematic to compute the CRC of a string that // contains…
Myrfy
  • 575
  • 4
  • 11
5
votes
2 answers

Getting data inside a google Chrome IndexedDB from Bash or Python

I have LevelDB (IndexedDB) file from my Google Chrome, the file is located in this folder: /home//.config/google-chrome/Default/IndexedDB/https__0.indexeddb.leveldb/ The folder content is: $ ls 000005.ldb 000006.log CURRENT LOCK LOG…
ccamacho
  • 707
  • 8
  • 22
5
votes
1 answer

A weird error in in LevelDB while compiling a custom layer of caffe

#include #include #include ... int arg_offset = 0; leveldb::DB* db1; leveldb::Options options; options.error_if_exists = true; options.create_if_missing = true; options.write_buffer_size =…
Khurram Amin
  • 172
  • 1
  • 11
5
votes
1 answer

Can you *read* leveldb data without the proper comparator?

I'm trying to access leveldbs generated by Chrome storing indexeddbs. I get keys and values. But they are either in an unknown encoding - I've tried many ways to detect them - or they are scrambled in some way. import plyvel db =…
Dej
  • 271
  • 3
  • 10
5
votes
1 answer

How to delete data older than some days in rocksdb or leveldb ?

I want to create a tool to delete all of the data older than some days in rocksdb or leveldb, but i don't know how to start it.
Michael
  • 367
  • 5
  • 12
5
votes
1 answer

why table and tablebuilder in leveldb use struct rep?

Recently I read the source code of leveldb, but I am confuse about the rep struct in the table and table_builder source. since we can directly store the member variable directly in the class Table and class TableBuilder. But why the author make a…
baotiao
  • 775
  • 6
  • 20
5
votes
1 answer

Default Comparator for LevelDB

I'm using LevelDB from Java via JNI. I want to supply a numeric (integer) key, and be able to iterate the db in the order of that key. Where I'm having difficulty is understanding how LevelDb's default comparator actually works, and how I would…
Kong
  • 8,792
  • 15
  • 68
  • 98
5
votes
1 answer

Thread-safety of leveldb snapshot

Is reading snapshot a completely thread-safe operation of leveldb? Specifically, is it thread-safe that one thread reads a snapshot of a leveldb database, while another thread is reading/writing on the same database? And what about another thread…
updogliu
  • 6,066
  • 7
  • 37
  • 50
5
votes
2 answers

Get All Keys From a LevelDB Database

I'm writing a script to collect the hashes of all bitcoin blocks. The program bitcoind, if a certain setting is changed, stores metadata for all blocks in a LevelDB database. The key of each set of metadata is the block's hash, which is typically…
Mike
  • 1,569
  • 2
  • 14
  • 20
5
votes
2 answers

Why use macro in the class declaration

I am reading the source code of leveldb, esp. regarding mutexlock. I found this declaration: class SCOPED_LOCKABLE MutexLock { public: explicit MutexLock(port::Mutex *mu) EXCLUSIVE_LOCK_FUNCTION(mu) : mu_(mu) { this->mu_->Lock(); } …
storm
  • 395
  • 1
  • 4
  • 11
5
votes
3 answers

Using key-value databases as a set with persistent indices

Since the below got a bit long: Here's the tl;dr; version: Is there an existing key/value best-practice for fast key and value lookup, something like a hash-based set with persistent indices? I'm interested in the world of key-value databases and…
milianw
  • 5,164
  • 2
  • 37
  • 41
1 2
3
…
26 27