6

I need to store around 100 millions of records on the database. Around 60-70% of them will be deleted daily and same amount of records are inserted daily. I feel a document database like Hbase, Big Table would fit in this. There are many other data stores like Cassandra, MongoDb, etc. Which data store would be useful for this kind of problem as there will be huge amount of reads/writes(order of 10's of millions) daily.

sravan_kumar
  • 1,129
  • 1
  • 13
  • 25
  • Perhaps you wanted a efficiency comparison. You may see each of these data-stores performances on their websites. We use Cassandra for similar situation, but I haven't played with other datastores – Nishant Dec 23 '11 at 08:14
  • the nature of your data probably affects this database comparision as well – Prescott Dec 23 '11 at 08:21
  • a lot of data stores can handle 100 million of records and these read/delete rates. What's the size of the records? what are your data access patterns etc. write vs. read ratios etc.? – Arnon Rotem-Gal-Oz Dec 24 '11 at 13:24
  • @ArnonRotem-Gal-Oz: The size of each record could be ~100KB, and read-to-write ratio would be 24:25 on non-weekend and 5:7 on weekends. – sravan_kumar Dec 26 '11 at 07:05
  • @sravan_kumar what about accessing the data ? by key? complex queries? – Arnon Rotem-Gal-Oz Dec 26 '11 at 19:29
  • @ArnonRotem-Gal-Oz: The access is by key-value pair. The value is actually a json object being stored.. I do not need column orientedness which Hbase and cassandra have. Will be only using Key-value feature.. – sravan_kumar Dec 27 '11 at 07:24

1 Answers1

4

Based on the characteristics you've mentioned (JSON Documents, accesses by key, 100 million records, balanced read/write) I'd say CouchDB or Membase are good candidates (here's a quick comparison)

Both HBase and Cassandra can probably also work but for HBase you'd need to install a lot of components (Hadoop, ZooKeeper etc) that you won't really use d only use and Cassandra is better when you have more writes than read (at least the last time I used it).

Big Table, is unfortunately internal to google : )

Arnon Rotem-Gal-Oz
  • 25,469
  • 3
  • 45
  • 68