2

I'm looking for a data store that is very memory efficient while still allowing many object changes per second and disregarding ACID compliance for the last X records.

I need this database for a server with not much memory and I can make a key-value store, document, or SQL database work.

The idea is that indexes/keys are the only thing I need in memory and all the actual values/objects/rows can be saved on disk do to the low read rate (I just want index/key lookup to be fast). I also don't want records constantly being flushed to disk, so I would like the last X number of records to be held in memory so that 100 or so of them can all be written at once. I don't care if I lose the last 10 seconds worth of objects/values. I do care if the database as a whole is in danger of becoming corrupt.

Is there a data-store like this?

Xeoncross
  • 4,449
  • 12
  • 43
  • 56

3 Answers3

2

Redis can handle this situation -- it writes everything to disk (eventually), but you can specify how frequently it does this, and anything not written is (naturally) lost.

http://redis.io/topics/persistence gives you all the gory details.

womble
  • 96,255
  • 29
  • 175
  • 230
  • The problem is that Redis requires the **data** to be stored in RAM also (or swap space). You cannot have a Redis database with more memory than RAM + Swap. It's the same problem that memcached has. I can't fit 2GB of data in a membase or redis database on a 256MB VPS. – Xeoncross Jul 08 '11 at 17:09
  • Hmm, I misinterpreted your question then. – womble Jul 08 '11 at 22:53
  • Redis comes the closest. If only they didn't keep the data in memory. – Xeoncross Jul 09 '11 at 01:38
1

Yes - any database using asynchronous I/O on a modern operating system will have near optimal caching.

symcbean
  • 21,009
  • 1
  • 31
  • 52
1

Redis has already been mentioned

Other options:

  • scalaris (somewhere on code.google.com) -- the complete CAP and is distributed
  • riak (basho.com -- eventually consistent, distributed)
serverhorror
  • 6,478
  • 2
  • 25
  • 42
  • Scalaris looks nice, but it [doesn't store data on disk](http://code.google.com/p/scalaris/wiki/FAQ) so you would need a node network setup if you didn't want to lose your data. – Xeoncross Jul 08 '11 at 17:26