4

I need a recommendation for a key-value store. Here's my criteria:

  1. Doesn't have to be persistent but needs to support lots of records (records are small, 100-1000 bytes)
  2. Insert (put) will happen only occasionally, always in large datasets (bulk)
  3. Get will be random and needs to be fast
  4. Clients will be in Ruby and, perhaps Java
  5. It should be relatively easy to setup and with as little maintenance needed as possible
leventov
  • 14,760
  • 11
  • 69
  • 98
Mladen Jablanović
  • 43,461
  • 10
  • 90
  • 113

4 Answers4

6

Redis sounds like the right thing to use here. It's all in memory so it's very fast (The GET and SET operations are both O(1)) and it supports both Ruby and Java clients.

Jeff Foster
  • 43,770
  • 11
  • 86
  • 103
  • Any experience on how does it handle bulk inserts (for example, will I be able to query it while inserting millions of records)? – Mladen Jablanović Jun 08 '11 at 14:25
  • You'll definitely be able to query whilst inserting - everything's in memory so there's no disk thrashing. It also features pipelining (http://redis.io/topics/pipelining) that means you can stream off bulk inserts without waiting for each individual result to come back – Jeff Foster Jun 08 '11 at 14:27
  • Just playing with Redis with some real data. Performance is really great, the only problem we have so far is occasional crashing, which seems to be happening when the memory gets exhausted. We may have to play a bit with Redis and box settings. – Mladen Jablanović Jun 09 '11 at 18:11
  • 2
    We are now successfully using Redis. Crashes were due to ~3GB memory limit for 32-bit version, moving to 64-bit solved the issue. – Mladen Jablanović Jul 14 '11 at 14:33
5

Aerospike would be a perfect because of below reasons:

  1. Key Value based with clients available in Java and Ruby.
  2. Throughput: Better than Redis/Mongo/Couchbase or any other NoSQL solution. See this http://www.aerospike.com/blog/use-1-aerospike-server-not-12-redis-shards/. Have personally seen it work fine with more than 300k read TPS and 100k Write TPS concurrently.
  3. Automatic and efficient data sharding, data re-balancing and data distribution using RIPEMD160.
  4. Highly Available system in case of Failover and/or Network Partitions.
  5. Open sourced from 3.0 version.
  6. Can be used in Caching mode with no persistence.
  7. Supports LRU and TTL.
  8. Little or No maintenance.
Ankur Choudhary
  • 485
  • 1
  • 9
  • 18
0

An AVL-Tree will give you O(log n) on insert, remove, search and most everything else.

Eduard Thamm
  • 942
  • 1
  • 6
  • 7
0

1 and 3 both scream a database engine.

If your number of records isn't insane and you only have one client using this thing at the same time, I would personally recommend sqlite, which works with both Java and Ruby (also would pass #5). Otherwise go with a real database system, like MySql (since you're not on the Microsoft stack).

Blindy
  • 65,249
  • 10
  • 91
  • 131
  • 1 and 3 both are well supported by popular nosql solutions, not sure why RDBMS would be more adequate. Also, the number of records is quite big (millions) and there will also be concurrent reads. – Mladen Jablanović Jun 08 '11 at 14:06