Is Couchbase a kind of storage that address GroupBy-based read and write of 4TB worth of data with low latency? If not, what size of data Couchbase is good for for low latency access ?
Asked
Active
Viewed 605 times
1
-
Do you mean 4TB for your dataset? What size are the data items. Couchbase has a 20MByte per item max default. – Matt Ingenthron Mar 09 '12 at 01:46
1 Answers
2
Couchbase can definitely handle 4TB of data. It will be fast to the degree you can keep your working set in RAM. So you can have disk greater than memory, but you want to have a really small # of cache-miss rates, which we let you monitor. If you see that % get too high, it is time to grow your cluster so that more ram becomes available.
4TB should be a few tens of nodes. At that scale, disk throughput starts to be the limiting factor (eg slow disks take too long to warm up lots of ram). So for really hot stuff, people use SSDs, but for the majority of apps EC2 is plenty fine.
J Chris A
- 2,158
- 15
- 14
-
1What if your primary concern is parallel write speed of new entities into the DB? How fast should I expect to insert 4TB of data into a say a 50 node ChouchDB cluster? 1 min or 1 hour? Is this time very different if I use MonogoDB? – iCode Feb 13 '13 at 00:53